Colyear was the son of David Colyear, 1st Earl of Portmore and his wife Catherine Sedley, Countess of Dorchester, daughter of Sir Charles Sedley, 5th Baronet, MP, of Aylesford, Kent, and former mistress of James II.

[1] Milsington was returned as Member of Parliament for Wycombe on the Wharton interest at a by-election on 1 February 1726 but on account of the partiality of the returning officer, the election was declared void on 22 February.

[1] On 2 January 1730 Milsington succeeded to the peerage on the death of his father and vacated his seat in the House of Commons.

He was a founding Governor of the Foundling Hospital, a charity created in 1739, dedicated to the salvation of abandoned children.

On 7 October 1732, Lord Portmore married Juliana Osborne, Duchess of Leeds, daughter of Roger Hele of Halwell, Devon, and widow of Peregrine Osborne, 3rd Duke of Leeds[4] Together, they were the parents of: Lord Portmore also had an illegitimate daughter by Elizabeth Collier: Lord Portmore died on 5 July 1785.