He was second son of Lieutenant-General George FitzRoy, 2nd Baron Southampton, by his second wife Frances Isabella, daughter of Lord Robert Seymour.

[1] His grandparents were Charles FitzRoy, 1st Baron Southampton and Anne Warren, daughter and co-heir of Adml.

FitzRoy was returned to Parliament for Great Grimsby in 1831, a seat he held until 1832,[2] and later represented Lewes between 1837 and 1841 and between 1842 and 1859.

He served under the Earl of Aberdeen as Under-Secretary of State for the Home Department between 1852 and 1855 and under Lord Palmerston as First Commissioner of Works between June and December 1859.

FitzRoy died on 17 December 1859 and was buried at City of Westminster Cemetery, Hanwell along with his wife.