He was the youngest son of Frederick Hervey, 1st Marquess of Bristol and Elizabeth Albana (1775–1844).
His maternal grandfather was Clotworthy Upton, 1st Baron Templetown.
Active in the affairs of the town, he was a founder Vice President of Brighton College and served on its Council from 1845 to 1875.
[1] He was commissioned as a Lieutenant in the disembodied West Suffolk Militia (commanded by his elder brother the 2nd Marquess) on 4 July 1831,[2] and was promoted to captain on 9 April 1852.
[1] Their eldest son Reverend Canon Frederick Alfred John Hervey (1846–1910) was Chaplain-in-Ordinary to Queen Victoria from 1886 to 1901 and Domestic Chaplain to King Edward VII from 1878 to 1910.