The site was originally occupied by Holly Hedge House, a building built in the 18th century for the younger sons of the Earls of Dartmouth.

[1] The site became the headquarters of the 3rd Kent Volunteer Rifle Corps, a unit which became the 20th Battalion, London Regiment (Blackheath and Woolwich) in 1908.

[1] During the Second World War the original house was badly damaged by bombing, and it was subsequently demolished in 1946.

[1] After the war the purpose-built drill hall was enlarged to accommodate better messing facilities and to incorporate the staircase from the old house.

[4] The drill hall was instead occupied by a rifle company of the 10th Battalion the Parachute Regiment.