Originally built from wood by the Hilton (later Hylton) family shortly after the Norman Conquest in 1066, it was later rebuilt in stone in the late 14th to early 15th century.

[1] The castle underwent major changes to its interior and exterior in the 18th century and it remained the principal seat of the Hylton family until the death of the last Baron in 1746.

They depict the coats of arms belonging to local gentry and peers of the late 14th to early 15th centuries and provide an approximate date of the castle's reconstruction from wood to stone.

[5][6] At this time, Adam de Hylton gave to the monastery of Hartlepool a pyx or crucifix, weighing 25 ounces (710 g) in silver and emblazoned with his coat of arms – argent, two bars azure.

[6] In gratitude, the king granted the eldest son, Henry, a large tract of land on the banks of the River Wear.

It was subsequently re-built in stone by Sir William Hylton (1376–1435) as a four-storey, gatehouse-style, fortified manor house, similar in design to Lumley and Raby.

[11] The household inventory taken on Sir William's death in 1435 mentions, in addition to the castle, a hall, four chambers, two barns, a kitchen, and the chapel, indicating the existence of other buildings on the site at that time.

[8] He also inserted large, alternating, pedimented sash windows in the Italianate style and added a three-storeyed north wing to the castle (as seen in Bucks' engraving of 1728).

Above the doorway was a coat of arms, believed to be the one created to commemorate the marriage between John Hylton and his wife, Dorothy Musgrave.

[10] When the 18th and last "baron" died without male heirs in 1746, the castle passed to his nephew, Sir Richard Musgrave, Bt, who took the name of Hylton.

[7][16] No record of her, or any of her family, ever taking up residence exists and the castle later passed to her grandson, John Bowes, 10th Earl of Strathmore and Kinghorne.

[8] William Howitt's Visits to Remarkable Places (1842) notes the rooms had "stuccoed ceilings, with figures, busts on the walls, and one large scene which seemed to be Venus and Cupid, Apollo fiddling to the gods, Minerva in her helmet, and an old king".

[19] By 1844, the chapel was used as a carpenter's workshop, and according to the Durham Chronicle in January 1856, the castle set on fire while in the occupation of a farmer, Mr.

[19][21] In 1862, the castle was put up for sale by the Strathmores and purchased by William Briggs, a local timber merchant and ship builder.

[3][10] Due to the advanced decay of the 19th-century alterations, the ministry removed all internal partitions and consolidated the shell to reveal the remaining medieval masonry.

[8] Bucks' engraving of 1728, shows a short nave and a large six-light west window, and that the chapel was disused by this time, as it had no roof.

[31] Although repairs to the chapel were carried out by the last Baron Hylton and the successive owners in the 19th century, it fell into disrepair until, like the castle, it was taken over by the state in 1950.

[10] The castle and chapel have been Grade I listed buildings since 1949 and form a Scheduled Ancient Monument under the care of English Heritage, who took over the site in 1984, although Sunderland City Council own the land.

[35] The screen closing off the east entrance has a three-bay cusped arcade on the ground floor and three ogee arches on the shafts above.

[36] Between the central towers once stood a sculpture of a knight in combat with a serpent (of which only fragments survive), believed to pertain to the tale of the Lambton Worm.

[21][36] The parapet is also machicolated (except on the north façade) and continued between the central towers by a carved-foliage arch (originally with cusping which fell in 1882), instead of corbels.

[8][11] Another feature of the roof was shallow stone troughs on the battlements which fed scalding oil or water into the machiocaltions as a means of defence.

[36] Before the changes made by John Hylton (died 1712), the castle's layout plan was as follows: The ground floor, accessed directly from the outside courtyard, led into a portcullis-protected, vaulted passage, eleven feet wide and extending the depth of the building.

[35][38] The oratory was entered via a five-and-a-half high pointed-arch doorway and contained an altar and piscina, of which only an ornamental niche remains.

[36][37] Above, on the roof level, was the Warder's Chamber containing a stone-hooded fireplace, beamed ceiling, two small windows in the east wall and a garderobe.

They are believed to show the political alliances of the early Hyltons, as the banner of the king, and the arms of nobles and knights of Northumberland and the County Palatine of Durham are shown.

[27] Nevertheless, the arms give a date for the construction and completion of the castle as between 1390 and the early 15th century, due to the following reasons: The east façade of the castle features a slanted shield containing the Hylton arms (Argent, two bars Azure) and a white hart (male deer), lodged, chained and collared with a coronet, Or.