[1] The building with its classical portico is in the style of a Palladian villa and is similar in design to its near-contemporary in the United States, the White House.

The house was designed by Lieutenant Samuel Russell of the Madras Engineers and construction began in 1803.

[1] Kirkpatrick built the mansion for himself and his Indian wife Khair un Nissa, who bore him two surviving children who were sent to England by the age of five and never saw them again due to the early deaths of their parents.

Within the compound is a miniature model of the building- legend has it that this was so Kirkpatrick's wife, who remained in purdah, could see the entire mansion, including the front.

During the Indian Rebellion of 1857, a group of rebels, led by Maulvi Allauddin and Turrebaz Khan, attacked the residency.