He became a tenant farmer and occasionally taught school; the Fillmore family's circumstances were so dire that they sometimes relied on the charity of their landlords to survive.

While living in Niles, he followed the advice of his wife and procured a clerk's position for their son Millard in the law office of Judge Walter Wood, who was also their landlord.

[3] Unhappy with trying to make the stony ground of their Vermont farms productive, they quickly grabbed the opportunity and moved to Cayuga County, New York, sight unseen.

[5] Duped, tired, and poor, Nathaniel eventually became a tenant farmer while occasionally teaching school, working the soil for landlords and taking their charity when necessary to survive.

[4] Over time, Nathaniel Fillmore's fortunes changed; he became prominent enough while living in Niles, New York that he served as a justice of the peace for eleven years.

[8][b] While living in Niles, Fillmore followed his wife's advice to secure a clerk’s position for their son Millard in the law office of their landlord, Judge Walter Wood.

[13] Questioned by a guest who wanted to know how to raise a son to become president, Nathaniel Fillmore alluded to his one-time poverty by replying "Cradle him in a sap trough.