It was originally a land office, expanded in two stages during the 19th century by its builder, Jacob Le Roy, an early settler for whom the village is named.

Le Roy expanded the small land office into a large house, with finely decorated interior.

During the late 19th century it was subdivided into a boardinghouse for faculty and students at Ingham and the Le Roy Academic Institute, an early secular private school.

Across the street is a small park and Le Roy's current elementary school , built in the early 20th century.

The house is a two-and-a-half-story stucco-faced limestone structure with a raised foundation and side-gabled roof shingled in asphalt.

[2] Its south (front) facade frames the centrally located main entrance in a hip roofed portico with classical entablature supported by paired fluted Doric columns with indented capitals.

All windows on the facade are tripartite, with 12-over-12 double-hung sash flanked by three-over-three sidelights on the first story complemented by eight-over-eight and two-over-two on the second.

The north elevation has tripartite windows like those on the opposite face surrounding the rear entrance, a six-panel door with narrow six-light sidelights and four-light transom.

[2] Paired paneled pilasters flank the six-light sidelights aside the main entrance, topped by another four-light transom.

The decoration is more restrained, with simpler door and window surrounds and a ceiling medallion in the north section of the hall made of concentric circles.

All rooms are accessed by six-paneled wooden doors with half-glazing consisting of hand-painted frosted glass depicting landscapes, ruins, birds or animals.

[2] The largest of the second floor bathrooms, the northeast one, has a fireplace with a Federal style mantel featuring classical entablature, inset molded panel and flanking pilasters.

Fenestration on that side consists of three aligned pairs of double-hung sash windows, one-over-one at the basement and first story, and two six-over-ones flanking a wooden door on the second.

[2] The paneled main entrance door, with single-light sidelights and transom, opens into a hall running along the west side.

In the wider central portion of the hall is the freestanding chimney, its breast faced in plaster, with turned wooden cornerbeads and beaded fir wainscoting.

A small storage room in the northwest corner is accessed from this space, divided from the main hall by a partially glazed wall on the south.

After the Revolutionary War, Robert Morris bought the land west of the Genesee River which now makes up Western New York from the state of Massachusetts.

Most of this he later sold to the Holland Land Company, but in 1793 the New York firm of LeRoy & Bayard acquired 87,000 acres (35,000 ha) from him.

Because of its shape, stretching from the current village of Le Roy to the shores of Lake Ontario and widening to the north, it was called the Triangle Tract.

[2] The intersection of Oatka Creek with a popular Iroquois trail was an obvious place to establish a settlement, and four years after the Triangle Tract was purchased a man named Charles Arthur built a log cabin on the north side of what is now Main Street, just east of the future village's municipal boundary.

[2] In 1813 the community was named after Herman Le Roy, senior partner in the landowning firm and former director of the Bank of the United States.

When the land was fully surveyed in 1817, he sent his nephew Egbert Benson Jr. to Le Roy to serve as the company's agent.

Jacob Le Roy, a son of Herman's who had traveled abroad learning the family business after studies at Yale, replaced his cousin as company agent.

He also oversaw extensive additions to the grounds, such as large barns, expanded gardens, and the replacement of a stone wall with iron filigreed gate posts.

[2] Citizens of the village, including Bartow, established the non-sectarian LeRoy Academic Institute in 1864 in a small building on Main Street.

Signs of former partitions remain on the floors of the large bedrooms upstairs, and the irregular fenestration of the side elevations also reflects this subdivision of the interior space.

During the 1880s the basement kitchen was remodeled, with the shiplap ceiling added and the original fireplace and bake oven bricked shut.

Its student population soon outgrew this space, and seven years later the current Le Roy Junior/Senior High School was built across Main Street and Trigon Park.

[2] In 1941 the local historical society was finally established and took title to the house, restoring the bake oven and brick hearth in the basement kitchen soon afterwards.

A new roof and gutter were installed in 1985, and Lexan and wire cloth were added to the basement windows to protect against damage from stray balls on the nearby softball field, also used for Little League.