[3] In light of Brunei's relation with the United Kingdom, the British consulate building's history extends back to the middle of the 19th century.

British Consul General Spenser St. John built a makeshift home in 1856 under Sultan Abdul Momin's rule, which was eventually renovated to a permanent building in 1858.

[6] Although excavation of the site would be necessary for physical proof, historical accounts indicate a direct connection between the current building and St. John's previous residence.

Though specifics are still unknown, estimates for 1940 included $3,500 for "alterations and repairs" to the residency, part of which might have gone toward a new garage and an addition on the building's southern side.

Wing Commander K. E. H. Kay served as the Senior Civil Affairs Officer during the brief British Military Administration occupation of the residency following the June 1945 landing of Allied forces at Muara Beach.

[15] The exterior and interior design of the building are based on a photograph and floor plan of the residency from 1948, which were confidently dated by State Engineer J. McLeod.

McLeod most likely created this concept in advance of significant modifications in 1949, some of which had already started in 1948 and included the residency of a guard house that is now a gardener's cottage and a second line of staff apartments.

Pretty's wife noted that a lot of work, such as re-roofing, re-flooring, and renovating the inside while maintaining its character, was required to make the house liveable.

With the exception of a tiny, unreliable Japanese generator, there was no electricity at the time, and supplies had to be shipped every week from Seria via rivers and beaches.

It is believed that in order to accommodate her huge party during Princess Marina's visit in the 1950s, the resident temporarily moved to a nearby home.

The residency's interior had additional changes in 1953, such as the removal of partition walls between the dining room and veranda to create a big living area.

With the sultan's participation, the Gilberts instituted a morning garden party custom that started with a march and ended with a reception at the residency.

It was paneled in stained wood to mimic a ship's cabin, and it had soft board ceilings, air conditioning, fitted wardrobes, and a larger toilet.

The guest bedroom, bathroom, and kitchen were updated during High Commissioner Fernley Webber's stay in 1967, and there were plans for a two-bedroom addition that never materialised.

[21] For Queen Elizabeth II's visit in 1972, the residency underwent extensive modifications, which included repositioning the store to make the pantry easier to reach and transferring the kitchen to a former guest room.

The veranda's electrical, plumbing, and drainage systems were renovated, and plywood paneling was installed in lieu of the open latticework at the back.

Perched atop Residency Hill, 3 kilometres (1.9 mi) south of Bandar Seri Begawan, this ideal site overlooks Kampong Ayer.

Despite changes made for structural support and size alterations brought about by site limits, it still largely resembles the original design from when it was built.

The main entrance's flat roof opens onto a veranda that encircles the front of the structure and has ornate porch railings with Southeast Asian-inspired designs.

Historical records show that the original structure featured two bedrooms and a central living room, however there have been significant internal alterations.