During his upbringing, he was taught philanthropy and public leadership by his father and his uncles who were involved with such work.

He attended Brown University (renamed in honor of a gift made by his father in 1804) and graduated in 1816.

When his collection became too large, he expanded his house, the Nightingale-Brown House, by adding a modern fireproof library; he also hired a full-time librarian, John Russell Bartlett, to manage the collection and produce its first catalogue.

He played major roles in Anti-Slavery campaigns, he became President of the Emigrant Aid Society.

[2] When his oldest son died in 1900, his well-collected books were granted to Brown University with an endowment and a building.