Built as the first Smithsonian museum building, it is constructed of Seneca red sandstone in the Norman Revival style (a recalling of a 12th-century combination of late Romanesque and early Gothic motifs; built in the Gothic and Romanesque revival styles).

The redstone was substantially less expensive than granite or marble, and while initially easy to work, was found to harden to a satisfactory degree on exposure to the elements.

A structural collapse in 1850 of partly completed work raised questions of workmanship and resulted in a change to fireproof construction.

"[8] Despite the upgraded fireproof construction, a fire in 1865 caused extensive damage to the upper floor of the building, destroying the correspondence of James Smithson, Henry's papers, two hundred oil paintings of American Indians by John Mix Stanley, the Regent's Room and the lecture hall, and the contents of the public libraries of Alexandria, Virginia and Beaufort, South Carolina, confiscated by Union forces during the American Civil War.

[3] Around 1900, the wooden floor of the Great Hall was replaced with terrazzo and a Children's Museum was installed near the south entrance.

[3] The Enid A. Haupt Garden was dedicated in 1987, along with the Renwick Gate facing Independence Avenue, built from Seneca redstone retrieved from the demolished D.C.

[12] Renwick designed the Castle as the focal point of a picturesque landscape on the National Mall using elements from Georg Moller's Denkmäler der deutschen Baukunst.

As constructed, the central section contained the main entry and museum space (now the Great Hall), with a basement beneath and a large lecture room above.

The East Wing contained storage space on the first floor and a suite of rooms on the second as an apartment for the Secretary of the Smithsonian.

[3] The plan allowed for expansion at either end, a major reason for the informal medieval-inspired design, which would not suffer if asymmetrically developed.