[1] His father was a banker and supporter of Union Theological Seminary and his paternal grandfather was Alexander Brown of Baltimore.

[3] Brown was educated privately and then entered Columbia University, where he graduated in 1859.

Brown served on the board of education of New York City, and was a trustee of Columbia University.

[1] On November 9, 1864, Brown was married to Mary Elizabeth Adams in New York City.

[6] Together, they were the parents of six children:[1] He died on June 25, 1909, in Orange Mountain House, New Jersey, and was buried at Green-Wood Cemetery in Brooklyn.