Having seized York in that year, William the Conqueror built a castle on the south side of the city close to the River Ouse.
A flight of steps led to a wooden structure at the top which was surrounded by a fence, also made of wood.
By c. 1340 part of the city wall had been built along the south-east and south-west sides of the Old Baile, incorporating the existing ramparts and ditch, however, these defences were rarely called into use.
The only notable occasion was the siege of York in 1644 during the Civil War, when Baile Hill was used as a royalist gun emplacement.
The only other visible evidence of the former castle are two slight dips in the city wall rampart, one next to Baile Hill and the other close to Victoria Bar, which indicate the location of the former ditch.