The surviving gatehouse resembles the architecture of Caerphilly Castle, which may indicate that the same master mason worked on both fortifications.

The Bishops Palace was intact in 1610 when it was shown in a plan by John Speed, but was probably destroyed during the English Civil War.

[7] The Bishop's Palace was really a small medieval fortress set in a rectangular enclosure measuring 52 by 40 m (170 by 130 ft).

It has two towers and originally had two storeys above the basement, a vaulted archway, a portcullis slot and a great oak door with much decorative ironwork.

The Bishop's Palace buildings are Grade I listed,[8] meaning they are of exceptional architectural and historical interest.