The manor of Penhow was held by Caradog ap Gruffydd, prince of Gwent at the time of the Norman invasion of Wales.

[2] Viscount Rhondda, an industrialist and conservator of ancient buildings in Wales, bought the castle in 1914.

[3] By the mid-20th century, the castle was in a state of some dilapidation, until bought and restored by the film director Stephen Weeks.

[a][7][4] The architectural historian John Newman, in his Gwent/Monmouthshire Pevsner, describes Penhow as "small and [un]convincingly defensive".

[8] Further large-scale remodelling took place in the 17th and 18th centuries, when the castle was refashioned as a comfortable country house.