[1] The house is built of ashlar in three storeys to a square plan (9-bay x 8-bay) on a raised basement with advanced corner towers.

[3] This, along with further disagreements on the issue of mason work resulted in the legal case between the architect and Earl.

[3] David Bryce Jr was later commissioned to provide a three-storey pavilion and corridor block, but this was damaged by a bomb in 1940 and subsequently demolished.

The Earls of Fife moved out of Duff House in 1903, gifting the property to Banff Burgh in 1906.

Duff House is also home to the Dunimarle Library, a collection of rare books, which can be viewed by appointment.