He succeeded his uncle in the lordship of Colville of Culross 1849 and was elected a Scottish representative peer in 1851.

He was made a Knight of the Thistle in 1874[12] and created Baron Colville of Culross, in the County of Perth, in 1885, in the Peerage of the United Kingdom.

Their second son was Sir Stanley Colville, who rose to prominence as a Navy officer.

Another son, George, was the father of Sir Jock Colville, civil servant and memoirist.

Lord Colville of Culross died in July 1903, aged 84, and was succeeded by his eldest son, Charles.