[1] He was a son of Stephen Baker, a merchant who was one of the original stockholders in the Manhattan Company, founded in 1799.

[3] In 1850, Baker engaged as importer of woolen goods, and moved to Poughkeepsie, New York.

[4] After his term in office, he abandoned active business pursuits and lived in retirement until his death.

[5] Baker was married to Anna Mary Greene (1827–1903).

Together, they were the parents of:[1] While en route to California for his health, on a train near Ogden, Utah, Baker died on June 9, 1875, aged 55 years and 301 days.