Probably built in the third or fourth centuries AD, it formed part of the coastal defences of the eastern shore of Britain, and overlooked the mouth of the River Deben.

[4] The name Walton denotes a settlement and farmstead of Wealas, ("strangers") the Anglo-Saxon term for native Britons, or Celts who, over time, adopted the language and culture of the newcomers.

With Dunwich, Walton Castle is one of the two principal sites claimed in the Middle Ages for the location of Dommoc, the original bishop's seat of St Felix (Felix of Burgundy), first bishop of the East Angles, who arrived c. 630 AD in the reign of King Sigeberht of East Anglia.

[citation needed] At the time of the Norman Conquest, the manor of Walton was linked with that of Falkenham, a village overlooking the River Deben a little further inland.

In about 1170–80 Roger Bigod, 1st Earl of Norfolk, invited the monks of Rochester to found Walton Priory,[5] dedicated to St Felix, in the precinct of the Roman fort.

Hugh Bigod also held a castle at Walton, but was obliged to surrender it to King Henry II in 1157.

Henry then stationed a royal garrison there until 1173, when the rebel Earl of Leicester landed there to join the Bigods.

King John was staying at Walton Hall in 1200 when he ratified the original Town Charter of Ipswich.

Large ruins of the old hall remained in the eighteenth century, and the last major portion of wall fell during a high storm in the 1880s.

The parish church was largely rebuilt in the 19th century on the medieval ground-plan, including the tower unusually sited at the south-west end of the south aisle.

W. Castle stood on a sea-cliff; dated from the time of the Romans; was rebuilt by R. Bigod, and ruined by Henry II.

The building is early nineteenth century in date, built c.1800–32, is a timber framed hexagonal structure and is shown on the tithe map of 1840 Walton Hall was built in c.1740–1750 and rebuilt and extended in c.1799 as evidenced by a date stone which is initialled for the contemporary owner Anthony Collett.