Its most notable occupant was Asa Gray (1810–88), a leading botanist who published the first complete work on American flora, and was a vigorous defender of the Darwinian theory of evolution.

Subsequent occupants included botanist Thomas Nuttall and Harvard presidents James Walker and Jared Sparks.

Asa Gray purchased the house in 1842 and moved in during the summer of 1844,[3] after receiving an appointment to a professorship at Harvard that he would hold for 45 years.

Already a rising star in the world of botany, Gray in 1848 published The General of the Plants of the United States, which was not only groundbreaking for the content, but also in its presentation.

Gardner Cox (one of Allen's children and a well known artist in Boston, converted the attached carriage house into an art studio).

The downstairs room of the ell served as Asa Gray's study, and includes a number of wood-frame display cases lining one wall.