The work was composed of a long brick revetted dry ditch running between a fortified guardroom on the Rochester-Maidstone Road to a similar tower alongside the Medway.

The principal work (still surviving) is a massive red brick keep, in the style of a medieval castle, which served as gun tower and observation post.

The dry ditch running across St Margaret's Street was crossed by drawbridge through a substantial casemated guardhouse in the form of an arch (which was demolished in the 1930s).

After nearby Fort Pitt became a military hospital, the patients were moved from Clarence to a new asylum, although the prison remained, with accounts of floggings being given in local newspapers.

Below the gardens — donated to the city of Rochester by former mayor Charles Willis in memory of a son killed in the First World War — is a sally port with sealed-up door.