A Grade I-listed building, the oldest parts of the complex date to 1708, but it was largely reconstructed in the 1860s because of termite damage.
It does not have the appearance of a typical castle, though historically the site was part of the East India Company fortifications of Jamestown.
The fort was rebuilt over the next several years to extend across the entire seaward mouth of James Valley and the Castle was immediately behind it to the east.
It was a modest one-storey building with a low basement; an upper storey was added in 1776.
Also part of the complex are the Printing Office, Savings Bank and the colony's archives.