He had Joseph Badger of Sheffield make substantial alterations and an addition of the west and east ranges of Renishaw Hall between 1793 and 1808.
His brother Francis inherited Barmoor Castle from their father's cousin, Samuel Phipps, and engaged architect John Paterson of Edinburgh in 1801 to build the present substantial castellated Gothic revival mansion there, incorporating some existing stonework of the old house.
Before her death on 3 May 1797, they were the parents of:[3] Sitwell married, secondly, to Sarah Caroline Stovin (1779–1860), on 23 July 1798 at Warmsworth, Yorkshire.
[1] Sir Sitwell died "of gout in the head" on 14 July 1811 and was succeeded by his only son, George.
After his death, his widow married John Smith Wright of Rempstone Hall, Nottinghamshire, on 19 August 1821.