His parents were the Reverend Hon John Evelyn Boscawen, Canon of Canterbury, (1790– 12 April 1851) and Catherine Elizabeth Annesley (died 30 July 1859).

On his marriage to Mary Frances Elizabeth Stapleton (24 March 1822 – 20 November 1891) on 29 July 1845, he acquired Mereworth Castle, near Maidstone, Kent.

[5] In 1845 following his marriage, he had access to Mereworth Castle in Kent which became the site of a large and successful stud, breeding Thoroughbred horses, and winning many classic races.

Archer is described by the National Horseracing Museum as ″... the best all-round jockey that the Turf has ever seen ...″, winning 2,748 races including 21 Classics – over half were for Lord Falmouth.

The gardens are first recorded in 1695 by the traveller, Celia Fiennes, in her book Through England on a Side-Saddle, when she visited Hugh Boscawen, a relative through marriage.