After the Battle of Hastings in 1066 William led his army to Canterbury and then sacked towns along the Pilgrims' Way, including Guildford.

The main bailey buildings would have included a great hall, apartments for the King and Queen and their chapels.

The great hall is thought to have been located at the site of the two houses at the bottom of Castle Hill and was made of stone.

King Henry had his room painted green with gold and silver stars and he also built a garden surrounded by marble columns.

[3] Henry purchased some extra land in 1245 to extend the bailey so allow him to build a set of room for Edward, his son and heir to the throne, which were completed in 1246.

[3] The keep is thought to have been heightened during the civil war which took place during Stephen's reign (1135–1154) and during the rebellion of Henry II's sons (the revolt of 1173–1174) the castle was strengthened.

[12][13] In 1367 (the 41st year of the reign of Edward III), it was given to Andrew Sackville, sheriff of Surrey and Sussex both for a prison and as a dwelling.

His family were involved with the castle for the rest of the 16th century and is thought to have been they who added the brick windows and fireplaces.

[18] In 1885 Lord Grantley of Wonersh, who owned a large part of the castle, sold it to the Guildford Corporation.

The tower and walls were restored and opened to the public as pleasure gardens in 1888 to mark Queen Victoria's Golden Jubilee of the previous year.

[1] The gardens are "extremely popular, displaying an amazing array of colourful bedding, centred on the 12th-century Castle Keep",[1] and include a life-size statue of Alice Through the Looking Glass, which is a memorial to Lewis Carroll who stayed nearby in The Chestnuts, his sisters' house, from 1868 until he died in 1898.

The keep now contains a visitor centre, open between April and September, that includes written panels describing the history of the castle and a model of the place as it appeared in about 1300.