With his wife Lady Jane Ogilvy, he was largely responsible for the foundation of Baldovan Institute in 1852,[4] Scotland's first residential hospital for learning disabled children.

[5] He was closely involved, along with Dr James Arrott, the head physician, in the moving of Dundee Royal Infirmary to a new site, and played a prominent part in the laying of the foundation stone for the new building on 22 July 1852.

On 7 July 1831 he married Juliana Barbara, the youngest daughter of Lord Henry Howard-Molyneux-Howard, at St George's Hanover Square.

[11] The couple had two children before Lady Juliana's death on 27 December 1833(1833-12-27) (aged 21):[2] Ogilvy remarried on 5 April 1836, to Jane Elizabeth Howard, daughter of Thomas Howard, 16th Earl of Suffolk, in Charlton, Wiltshire.

Sir John died on 9 March 1890(1890-03-09) (aged 86) in Archerfield, East Lothian,[11] whilst on a visit to his son Henry Hamilton Ogilvy accompanied by Eveline, his youngest daughter.