His siblings Mary, Edith, Ada, Reginald, William, Anna and Evelyn all moved, either to different parts of the United Kingdom or to the Colonies, including British North America.

George Smith Drew, was born 22 October 1818 in Kensington, then a village on the outskirts of London, to an affluent family.

Julius' grandfather, George H. Drew, was born in Belgravia, in Westminster in 1790 but at the age of four moved with his parents to their country house in the South Hams, Devonshire.

George moved back to London aged twenty-one and became a tea merchant during the birth of the Industrial Revolution.

[citation needed] The Drew family had close ties with the French aristocracy in the eighteenth century by the lineage of King Louis XIII of France through his great-granddaughter.

Thomas later migrated back to England with his father at a young age, at some time before the outbreak of the French Revolution of 1789,[citation needed] and became an architect based in Sloane Street, Westminster.

After only six years, in 1889, he and his partner, John Musker, were able to retire from active participation in the firm as extremely rich men.

Julius's first cousin was Richard Peek, the rector of Drewsteignton (named after Drogo de Teigne, alleged forefather of the Drewes).

Julius stayed on several occasions with his cousin and it must have been here that he conceived the idea of building a castle on the home ground of his ancestor.

According to his son Basil, he did so on the advice of William Hudson, proprietor of Country Life, who was both a patron and a champion of Lutyens.