It was built on the site of an Augustinian priory that was a daughter house of Notley Abbey in Long Crendon.
Jasper Tudor, Duke of Bedford bought and largely remodelled the priory in the 16th century.
In late 1771 the manor was bequeathed to Sir Francis Bernard, 1st Baronet on the death of his cousin.
He had served as governor of the provinces of New Jersey and Massachusetts Bay, where his uncompromising policies were instrumental in the events leading to the American Revolution.
[2] It was used as a filming location of Agatha Christie's Marple's Ordeal by Innocence in 2007.