It was built in 1745–55 as a folly in the form of a castle which incorporated office spaces and recreation rooms,[1] but may have originally been a stable block and laundry for the lord of the manor.

[3][4] Designed in Gothic Revival style, the building is symmetrical in plan with crenellated circular towers at each corner that link two-storey blocks to form a square courtyard.

[1] The front corner towers have external entrances with two-centred archways and above them are prominent white flushwork panels, some of which are in the shape of crosses while others mimic arrow slits.

[1] The Castle is built from pre-cast black copper-slag blocks from William Reeve's 'Bristol Brass Company' foundry at Crew's Hole.

[5] Much of the freestone carving and dressings on the building supposedly came from the city's demolished medieval gateways and St Werburghs Church, rebuilt by James Bridges in 1758–61.