Brown was born in New York City on December 29, 1865, and named after his maternal grandfather, the Rev.

[3] Brown graduated from St. Paul's School in Concord, New Hampshire before attending Yale University where he received an A.B.

[4] After returning to the United States in 1892, Brown joined the faculty of Union Theological Seminary as an instructor of Church History.

He was also instrumental in the founding of Union Settlement in East Harlem where he was largely responsible for fundraising.

The family lived at 49 East 80th Street in New York City and had a summer home on Mount Desert Island in Maine where they befriended many prominent people, including Charles W. Eliot, president of Harvard, Seth Low, president of Columbia and later Mayor of New York, and John D. Rockefeller Jr.