[2][3] It is the oldest structure on the island that is not a fortification,[3][4] but contrary to popular misconception, is not the Colonial Governor's Mansion, which was built in 1702.

[2] The Governor's House is a two-story Georgian brick structure.

The entrance portico contains Ionic columns under an entablature, with a paneled wooden door behind a transom, and is accessed by a concrete-upon-brick stoop.

[2] The windows around the house are six-over-six, double-hung, with brownstone windowsills.

The east-side annex has a Colonial Revival doorway and cast stone sills.