Construction began in 1852, but was interrupted in 1859 due to costs and the possibility of South Carolina's secession from the Union.

Noted New York architect, James Renwick submitted a late entry, which was returned.

The commission judging the entries selected the Jones design and submitted the plans to the Secretary of the Treasury in Washington, DC.

Eventually, Ammi Burnham Young was selected to produce a new design incorporating features of the four competitive entries.

The west and east arms had Roman porticoes supported by Corinthian limestone columns and steps down to grade.

[3][4][7][8] Except for the change from Doric to Corinthian order and for the high dome, the building bore resemblance to the Boston Customhouse that Young had recently designed.

Since the location was marshy, 7,000 piles that were 40 ft (12 m) long were driven into the sand and were cut off at grade.

[4] In 1859, there was increasing concern in Congress over the possible secession of South Carolina and the costs of construction.

[9] Alterations made at his direction included the loss of the dome, the elimination of the north and south porticoes, and changes to the west entrance.

Young's original design was replaced in 1876[11] with skylights that covered a two-story, square cortile or inside patio.

[3][5] It is possible that the north and south portico were enclosed to increase office space in repairs after the 1886 Charleston earthquake.

Threatened with demolition, local preservationists with the help of Representative Mendel Rivers worked to save the building.