In 1372 George de Dunbar, 10th Earl of March, gave in marriage with his sister Agnes to James Douglas of Dalkeith, the manor of Whittingehame, with the patronage of the chapel.

The Douglases remained in possession for over 200 years: about 1537 Elizabeth (d. after August 1557), daughter of Sir Robert Lauder of The Bass (d. 1517/18), married William Douglas of Whittingehame, and in October 1564 Mary, Queen of Scots, confirmed to their son, William Douglas of Whittinghame (d. 17 December 1595), a Senator of the College of Justice, the barony of Whittingham, the castle, mills, and the avowson of the Church there, ratified by parliament on 19 April 1567.

This William Douglas had married in 1566 Elizabeth (d. after 6 August 1608), daughter of Sir Richard Maitland of Lethington, a Senator of the College of Justice.

Whittingehame passed to his brother Sir William Douglas of Stoneypath, near Garvald, whose daughter Isobel married, in 1628, Sir Arthur Douglas of the Kellour family, and their daughter Elizabeth (1632–1668) married, in 1652, Alexander Seton, 1st Viscount of Kingston and carried Whittingehame to him (Elizabeth's brother Archibald having died unmarried).

A parochial school, of which the laird was patron, was long established at Whittingehame, and Mr James Hogg was appointed schoolmaster there in 1742; having transferred from neighbouring Morham.

In 1820, James Balfour rebuilt the church, supplanting the previous rebuild of 1722, and then established, in 1840, a new model village to the north-west of the former medieval settlement.