[7] The Potter Building's architect, Norris Garshom Starkweather, was known for designing churches and villas in the mid-Atlantic states.

[1] The Potter Building employed the most advanced fireproofing methods available at the time of construction, due to its predecessor having burned down.

[16] The Potter Building is U-shaped, with a "light court" within the two arms of the "U", facing outward toward Beekman Street.

[19] At the time of the Potter Building's construction, the facades of many 19th-century early skyscrapers consisted of three horizontal sections similar to the components of a column, namely a base, midsection, and capital.

[20][21] Each side has similar ornamentation, containing column capitals, pediments, corbels, panels, and segmental arches made of terracotta.

[6] The ornamental detail is elaborately designed in the classical style and includes massive capitals atop the vertical piers, as well as triangular and swans'-neck pediments.

They contain concealed flues that ventilate the gases from the building's furnaces into hidden chimneys underneath the finials atop each pier.

The lintels on the upper stories, clad with terracotta, consist of four parallel wrought-iron beams with a width of 21 to 32 inches (530 to 810 mm) between flanges.

[24] Because of the presence of elevator lobbies at the northern end of the building, the northernmost bays on Park Row and Nassau Street are wider.

[17][37][13][42] Potter erected a five-story Italianate stone building on the lot for $350,000 (equivalent to $11 million in 2023[b]); it became the first headquarters of the New York World, which was established in 1860.

[12][46] The fire destroyed much of the block within a few hours, killing six people[47] and causing more than $400,000 in damage (equivalent to $13 million in 2023[b]);[44][45] The World building was said to have "made itself notorious the country over for burning up in the shortest time on record",[10][12][37] and it took a week to examine the wreckage,[47] Several days after the fire, the Real Estate Record and Guide said that "the ground is so valuable that it will no doubt be immediately built upon".

[51] To test the relative strength of iron versus wooden floor beams, Potter built two small, nearly-identical structures, one with each material.

[7] Potter's involvement in the process of terracotta selection was so extensive that he founded the New York Architectural Terra-Cotta Company with his son-in-law Walter Geer.

[34] The 1892 King's Handbook of New York City stated that newspapers, magazines, insurance companies, and lawyers occupied 200 offices within the building.

In addition, Potter occupied the top floor, and his New York Architectural Terra Cotta Company also had offices in the building.

[55] The Seaman's Bank for Savings acquired the structure at a foreclosure auction in 1941,[59] and four years later, sold it to Beepark Estates.

These plans did not proceed and Pace sold the building in 1979 to a joint venture named 38 Park Row Associates,[55] composed of Martin Raynes and the East River Savings Bank.

The Potter Building's co-op board subsequently arranged for a renovation of the facade in 1992-1993, to be carried out by Siri + Marsik and Henry Restoration.

[35] The Potter Building, along with the Manhasset Apartments and 110 East 42nd Street,[63] was made a New York City designated landmark on September 21, 1996.

Residents of the Potter Building complained that the seating violated a city ordinance on sidewalk cafes.

[2] Lower Manhattan's late-19th century skyscrapers generally received mixed reception,[6] and the Potter Building was especially criticized by professional architectural journals.

[65] A Real Estate Record and Guide writer remarked in 1885 that "there is not an interesting or refined piece of detail in the whole building".

"[67] Later, in 1991, New York Times writer David W. Dunlap described the Potter Building as "almost hallucinatory in its Victorian encrustation".

[68] Architectural writers Sarah Landau and Carl Condit characterized the Potter Building as "distinguished above all by its ruggedly picturesque red brick and cast-iron-clad outer walls abundantly trimmed with terra-cotta".