William was the Lieutenant Governor General of New South Wales; which meant he was hypothetically 2nd in command to running Australia.
It is built of granite with sandstone dressing to the quoins and window surrounds, and there are two storeys together with an attic floor.
The building's most striking feature is its array of curvilinear parapeted gables topped by iron finials.
The house remained empty and gradually fell into decline until it was bought by the Morgan family.
This was a reference both to the shire of that name in which the property was situated and to General Sir Ralph Abercrombie, under whom William Stewart had served in the 1790s in the West Indies.
The house was sold in 1901 to the Good Samaritan Sisters and they wished to bring the gates from their old convent in Pitt St which was being demolished.
In the early 1900s Arthur Wigram Allen who was Harriet's nephew visited his relatives at Abercrombie house.
Arthur Wigram Allen was a prominent lawyer in Sydney but he is most remembered for the numerous photographs he took which are today held by the NSW State Library.
When he visited his Aunt at Abercrombie House in this Edwardian era he took some photos of the property which are shown.