While a schoolboy, at the age of 13, he wrote a manuscript history of Northampton, and from that time he was always engaged in enlarging his collections.
At the latter date, 220 of his original subscribers had failed him, and with health and means exhausted he was compelled to bring the publication to a close.
His library and manuscript collections were dispersed by auction in 1842, the latter passing into the possession of Sir Thomas Phillipps.
Baker, who was a unitarian, took a deep interest in various local institutions, and was a magistrate for the borough of Northampton.
A sister, Miss Anne Elizabeth Baker, was his constant companion for more than sixty years.