The title to the castle fell into the hands of the House of Neville after the 1349 Black Death, but was reclaimed by Robert Hansard in 1391.

[3] In about 1579 Thomas Jennison, Auditor General of Ireland (d.1579), purchased from the Ascough family the manor of High Walworth.

[11] In 1679 Francis Jennison sold the estate and emigrated to Europe,[4] possibly because in 1678 Thomas Jenison was accused of involvement in the Popish Plot to assassinate Charles II, arrested by Titus Oates and thrown into Newgate Prison.

The castle was searched for arms in 1689 in response to suspicions of a potential rebellion against the Protestants William III and Mary II.

), British Army officer of the 68th Regiment of Foot, later promoted to Lieutenant-General; the castle passed to the Aylmer family in 1819 at the death of John Harrison.

Vivian became High Sheriff of Durham and a big game hunter who explored Somaliland and crossed the Horn of Africa in 1885.

In World War II, during the Eades' ownership, the castle was used as a prisoner-of-war camp for 200 men including German and Italian officers, under the command of Major Rollin Holmes.

This Tudor castle, dating from around 1600, is a manor house built of partially rendered limestone rubble, and the roof is of Welsh slate.

[19] Internal renovation took place in 1740, so that the interior now has important mid−18th century features, such as Palladian plasterwork and Rococo details.

[6] The parkland south of Walworth Castle was originally enclosed demesne land, and as such there is still evidence of ridge and furrow fields.

[20] This field system may be associated with the lost settlement at North Farm, Walworth; however a possible enclosure has been identified in the park, close by.

[23] A medal apparently belonging to Thomas Jennison, the 16th-century builder of the castle, was found in 1937–1938 during road-widening work on Newton Lane.

The lodge and gate piers are both battlemented and the single-storey, L-shaped building has corner towers and rock-faced masonry with ashlar dressing.

[25] Huts to the east of the castle have been identified as the possible site of the World War II prisoner-of-war camp.

[39] In celebration of the Golden Jubilee of Elizabeth II in 2002, there was a children's fancy dress party at the castle in aid of charity.