About February 1561-2 he was compelled to resign the rectory of St. Andrew Wardrobe on account of his refusal to subscribe a confession of faith which Grindal, bishop of London, required from all his clergy.
Queen Elizabeth occupied the provost's lodge at King's College during her visit to Cambridge in 1564, and Baker was one of the disputants in the divinity act then kept before her majesty.
[2] In 1565 some of the fellows of the college complained against Baker to Nicholas Bullingham, bishop of Lincoln, their visitor: the provost was charged with neglect of duty in divers particulars, and with favouring popery and papists.
The bishop gave him certain injunctions, which, however, he disregarded: "By them the provost was enjoined to destroy a great deal of popish stuff, as mass books, couchers, and grails, copes, vestments, candlesticks, crosses, pixes, paxes, and the brazen rood, which the provost did not perform, but preserved them in a secret corner" in the belief that "that which hath been may be again".
[3]In 1569 the fellows again complained of him to Bishop Grindal and Sir William Cecil, chancellor of the university; and ultimately the queen issued a special commission for the general visitation of the college.