He had a younger brother, John, and three sisters, one of whom, Cecily, was the wife of Thomas Sackville, Lord Buckhurst.
He was elected a Member (MP) of the Parliament of England for Horsham in April 1554, Lancaster in November 1554, New Romney in 1555 and New Shoreham in 1558.
Baker married firstly Katherine Tyrrell, the stepdaughter of Sir William Petre.
She was the daughter and heiress of John Tyrrell (d.1540), esquire, of Heron in East Horndon, Essex, (eldest son of Sir Thomas Tyrrell and Constance Blount), and his wife, Anne Browne,[3][4][5] the daughter of Sir William Browne, Lord Mayor of London, by his second wife, Alice Keble (d. 8 June 1521), the daughter of Henry Keble (1452 - April 1517), Lord Mayor of London, by whom he had two sons and a daughter.
One of his daughters by Mary was Chrysogna Baker (d. 1616) who married Sir Henry Lennard, 12th Baron Dacre and had issue.