Railway service in Brewster can be traced as far back as December 31, 1848 when the New York and Harlem Railroad expanded their main line from Croton Falls to Dover Plains stations.

Realizing that the NY&H was going to run through the Town of Southeast, Walter and James Brewster constructed passenger and freight stations in 1848, and donated the buildings to the railroad.

Penn Central merged with the New Haven Railroad and its affiliates in 1969 giving them control of all lines in the village.

Penn Central's continuous financial despair throughout the 1970s forced them to turn over their commuter service to the Metropolitan Transportation Authority which made it part of Metro-North in 1983.

Smaller parking lots are located along eastbound US 6 and on a private road named Ellen Avenue, where it is also notable for having a grade crossing right next to the station, like Katonah.