Built on a hill known as Mount Fitzwilliam and completed in 1806, this imposing stuccoed-coral-rock building on Duke Street is the Bahamian archipelago's foremost example of Georgian Colonial architecture.

[2] The Oriental Herald, in 1825, stated: "The new Government-House, standing on the centre of the ridge that overlooks the town, was built by a sum voted by the House of Assembly from the funds of the Treasury and cost upwards of 20,000l.

[5][self-published source] This two-story incarnation measured more than 100 feet in length, and its primary façade, overlooking the harbour, was dominated by a full-length upper gallery supported by ten columns.

As a history recounts, "[T]he eastern wing of the Government House was unroofed on three sides and damaged to the extent that it was not fit for occupancy so major repair work had to be done to the roof and buildings".

[11][6] From late 1940 through 1941, Government House, then described as "a cracked and flaking edifice ... with about as much warmth and atmosphere as Wellington Barracks", was renovated by the Duchess of Windsor, whose husband, the Duke, formerly King Edward VIII of Great Britain, served as Governor of the Bahamas from 1940 to 1945.

The resulting decor was described as "'frankly smart and modernistic' with Regency touches" and included a room with baseboards made of natural rattan and a table whose surface was ornamented with a three-foot-long replica of the Duchess's signature.

The door also withstood the Duchess who was permitted to cover the upper half of the battered wood with a black glass panel on which is appropriately printed in white the famous device of the Duke's Order of the Garter—Honi soit qui mal y pense".