Historians suspect it was designed either by Murdoch Bruce, a Scotsman who was inspector of buildings, or by Lieutenant Bernard Collinson of the Royal Engineers.

[3] Construction was completed in 1846,[2] and the first occupant was Major-General George Charles D'Aguilar, General Officer Commanding from 1844 to 1846, who also held the post of Lieutenant Governor.

[3] The west and east wings were shelled during the Japanese invasion in the Second World War; it also suffered bomb damage.

It was handed over by the military to the civilian Hong Kong Government as part of the recovery of Victoria Barracks.

[5] It is named after local collector (and Vitasoy founder Dr. Kwee Seong Lo), who made a donation to the city in the 1970s.