He was born in London, the son of Samuel Baker, and was educated at the Collegiate School, Gloucester and at Cambridge University.

He entered the Indian Navy in 1845, taking part in a survey of the Arabian coast and the suppression of the slave trade.

In 1858, he left the army as a very junior subaltern, and was admitted to Magdalene College, Cambridge, being awarded a BA in 1862 and MA in 1865.

[1] During this time, Baker was much involved with the suggestion of a "military degree", a scheme supported by the Prince Consort, but lost on his untimely death.

Baker also devised a new plan for the mobilisation of the volunteer forces, having them concentrate on railway centres, and which was implemented in part.