Born on February 28, 1832, in Parma, Monroe County, New York,[1] Baker moved with his parents to what is now Fulton County, Ohio, where he attended and later taught in the common schools.

[2] He received an Artium Magister degree in 1879 from Wesleyan University in Delaware, Ohio.

[4] Baker was elected as a Republican from Indiana's 13th congressional district to the United States House of Representatives of the 44th, 45th and 46th United States Congresses, serving from March 4, 1875, to March 3, 1881.

[2] Baker was nominated by President Benjamin Harrison on March 24, 1892, to a seat on the United States District Court for the District of Indiana vacated by Judge William Allen Woods.

[1] He was confirmed by the United States Senate on March 29, 1892, and received his commission the same day.