He was the uncle of multi-talented sportsman Maxwell Woosnam, an Olympic and Wimbledon champion at lawn tennis and one-time captain of the England national football team.

They had children:[3] He lived at Middleton Hall in Northumberland, but in 1905 bought the estate of Stobo Castle, near Peebles, in the Scottish Borders.

A visit to Japan on the way back from the Test series in Australia in 1895 inspired him to apply knowledge that he acquired there to the estate of Stobo Castle.

[4] The Japanese-style water garden that Philipson created at Stobo between 1909–13 has been maintained, despite a succession of owners after the estate was sold shortly after his death.

The Stobo estate was acquired in 1971 by Leo Seymour, whose family still own it (but not the castle), and the Water Gardens are occasionally open to the public.