Some records state that there was a building of some construction on the site of the castle in 1130 and used by Roger De Mowbray.

[1] The earliest extant building on the site is the ruin of a quadrangular castle, probably begun in 1327 by Geoffrey Scrope, which was reputedly demolished in the Civil War.

They held it until 1768 when it was purchased briefly by William Chapman who sold it the Turton family.

[3] In August 1609, Stephen Proctor and Timothy Whittingham captured three priests, including John Mush and Matthew Flathers in the castle vaults.

The 19th-century writers William Grainge and John Gilbert Baker noted that he reportedly left a curse on any owner of Upsall who should prove disloyal to his king and country.