Born to a working class German immigrant family, his political fortune was a notable example of the effect of civil service reform.
His father fought in the Hanoverian Army at the Battle of Waterloo in 1815, immigrated to Baltimore, Maryland in 1819, and began preaching to various Seneca County, New York congregations in 1821,[1] and was also notable as a critic of Mormonism.
[1] Willers' early life was one of toil; he divided his time between working on the family farm in the summers and attending district school in the winter.
[1] By the age of 21, Willers was a Hard-Shell Democrat, supporting the election of Greene C. Bronson for governor in 1854.
[1] He was appointed to a clerkship in New York Secretary of State Gideon J. Tucker's office in 1857, and held this position during the succeeding terms of David R. Floyd-Jones and Horatio Ballard until 1863, when Governor Horatio Seymour appointed him as his personal secretary.