He was considered the father of rural mail delivery in the United States.

[7] In 1872, Baker and his brother Charles W. started a canning factory in Aberdeen.

He was elected to the Maryland Senate in 1893, after his brother John H. Baker withdrew his nomination and defeating Thomas H. Robinson, but only served from 1894 to 1895.

[1] He represented the Second Maryland Congressional District which included, at that time, Harford, Baltimore, and Carroll Counties.

Baker, however, was willing for the experiment to be made in his district, and the first rural route of the country was started in Carroll County, from Westminster post office.

[4][9] In 1868, Baker married Olivia Wells, of Aberdeen, who died after 16 months.

[3][2][5] Baker earned the nickname "the Grand Old Man of Harford" for his service in politics.

He was interred in Baker's Cemetery, the family burial ground, in Aberdeen.