Born in Webster, New York, Foster read law to enter the bar in 1859.
He was a member of the Kansas Senate from 1863 to 1864, returning to private practice in Atchison from 1864 to 1867.
He then served as Mayor of Atchison in 1867, returning to private practice in 1868, where he continued through 1874.
[1] On March 9, 1874, Foster was nominated by President Ulysses S. Grant to a seat on the United States District Court for the District of Kansas vacated by Judge Mark W. Delahay.
Foster was confirmed by the United States Senate on March 10, 1874, and received his commission the same day.