He was born Matthew Charles Howard Gibbon in London, and was the oldest son of Charles Howard, 11th Duke of Norfolk and Mary Ann Gibbon—his long-time mistress and purportedly his third wife.
Matthew was married to Charlotte Blackman in 1822, and they originally lived on a small estate in Yapton, but were not known to have had any children.
Matthew received his hyphenated surname and family coat of arms in 1842 by Royal Licence obtained by his brother, Edward Howard-Gibbon, with consent of the 13th Duke of Norfolk.
In 1869, he was noted as the last member of the College to actually reside there during his long tenure in that position.
He died 16 December 1873 in Yapton, and he left most of his estate to his niece and accomplished Canadian schoolteacher-artist Amelia Frances Howard-Gibbon.