It passed to Clan Baillie in the early 16th century[4] and then to George, 1st Lord Forrester in 1635.
The castle was purchased by Gordon Millar in 1957, who began to restore it, but he was only able to stabilise the stonework.
He set up the Torwood Castle Charitable Trust before his death in 1998, in order to continue the work.
In 2017 local councillor Niall Coleman asked Mr Andrew Bryce (in his capacity as a former trustee) to again become a standard trustee to save the castle, in lines with the constitutional document but Mr Bryce at this time declined in order to look after his wife.
In September 2018, Mr Gary Grant, a local man from Denny, moved into the castle, in order to protect it from vandalism, as it was not secured.