Early governors or their lieutenants resided at Mont Orgueil and later at Elizabeth Castle, although the latter's status as a tidal island meant that for convenience accommodation in the town of Saint Helier was sought.

At the time of the Battle of Jersey in 1781, Major Moise Corbet resided at Le Manoir de la Motte, where he was apprehended by the French invaders.

Around 1800, under General Andrew Gordon, a house adjacent to the Royal Square was acquired, described in 1809 as a "large, substantial and commodious stone mansion with appropriate offices, pleasure and kitchen gardens".

[1] Under General Sir George Don, the house was enlarged by the addition of offices for the conduct of public business,[2] but Major-General Sir Colin Halkett was petitioning the British Government in 1821 to agree the exchange of this town centre property on the grounds that the house needed expensive repairs, was prone to flooding from Le Grand Douet, the adjacent brook, and that "the rooms are ill calculated for public entertainment".

Major-General (later promoted Lt.-General) Halkett had written that the lieutenant governor "would at Belmont possess the desirable opportunity of seeing together, without apparent partiality, such of the inhabitants, and strangers, as naturally expected to be invited to Government House".