In 1796, they arrived in New York State and settled on a farm along Wappinger's Creek on land that had been part of the 1685 Rondout Patent near Manchester Bridge in Dutchess County.

While the farmhouse was being built, the family lived on the Filkintown Road, at what is now the intersection of Main and Church Streets.

In 1801, James Vassar brewed ale with barley grown from seeds his brother Thomas brought from Norfolk.

Demand for the ale was such that, in 1801, James Vassar sold the farm and bought a lot between Main and Mill Streets in the village of Poughkeepsie from Baltus Van Kleeck to build a brewery.

[2] In 1806, one day before he was to begin his apprenticeship, he ran away and crossing the Hudson River on the ferry at High Point made his way to Balm Town, just north of Newburgh, New York.

He subsequently took a better-paying job with another local merchant before returning to Poughkeepsie in 1810, where he joined the family brewing business as bookkeeper and collector.

Matthew, then only 18, took over management of the business which was then conducted out of part of an old dye house belonging to George Booth, husband of Vassar's sister Maria.

He spent his days at the brewery and his evenings working at an oyster saloon and restaurant he had opened in the basement of the county courthouse.

In September 1824, he was among those welcoming the Marquis de Lafayette to Poughkeepsie on the occasion of the General's visit to the United States.

He joined the board of the Farmers and Manufacturers National Bank and, in 1835, was elected president of the village of Poughkeepsie on the "Improvement" ticket.

When members of the Underground Railroad fell short of the amount necessary to buy Bolding's freedom, Matthew Vassar was among those who made up the difference.

[10] In 1864, Vassar purchased the art collection of Elias Lyman Magoon, a noted collector of Hudson River School paintings.

[5] Vassar's diary entry for June 16, 1865 states, "Sick and tired of College business, no one to help me Except "Scow", Doct Raymond and Swan.