Blazon Arms: Quarterly of six, 1st (Howard), Gules, a Bend between six Crosses-Crosslet finchée Argent; on the bend an Escutcheon Ar, charged with a Demi-Lion pierced through the mouth with an arrow, within a Double Tressure flory counterflory, all Gules, and above the escutcheon a Mullet sable for difference; 2nd (Thomas of Brotherton), Gules, three Lions passant guardant Or, and a label of three-points Argent; 3rd (Warrenne, Earl of Surrey), Chequy Or and Azure, 4th (Mowbray, Duke of Norfolk), Gules, a Lion rampant Argent; 5th (Dacre), Gules, three Escallops Argent; 6th (Greystock), Barry of eight Argent and Azure, three Chaplets of Roses proper.

Crest: On a Chapeau Gules, turned up Ermine, a Lion statant guardant with tail extended Or, ducally gorged Argent, charged on the shoulder with a Mullet for difference.

Lord Carlisle was the member of a junior branch of the Hay family, headed by the Earl of Erroll.

His third but eldest surviving son, the fourth Earl,[3] sat as Member of Parliament for Morpeth.

His eldest son, the tenth Earl, was Liberal Unionist Member of Parliament for Birmingham South.

Sir Charles Howard, fourth son of the third Earl, was a general in the Army and also represented Carlisle in the House of Commons for many years.

Charles Howard, Viscount Morpeth, eldest son of the fourth Earl from his first marriage, briefly represented Yorkshire before his early death from tuberculosis.

Frederick Howard, third son of the fifth Earl, was a major in the 10th Hussars and fought at the Battle of Waterloo in 1815, where he was killed in action.

Henry Edward John Howard, fourth son of the fifth Earl, was Dean of Lichfield.

Edward Granville George Howard, fourth son of the sixth Earl, was created Baron Lanerton in 1874.

Charles Wentworth George Howard, fifth son of the sixth Earl, represented East Cumberland in Parliament for almost forty years.

Lady Harriet Howard, third daughter of the sixth Earl, was the wife of George Sutherland-Leveson-Gower, 2nd Duke of Sutherland.

Geoffrey William Algernon Howard, fifth son of the ninth Earl, was a Liberal Member of Parliament.

The Earls of Carlisle are distantly in line to succeed to the Dukedom of Norfolk, as descendants of Lord William Howard.