In their business dealings, John and Harold relied upon George W. R. Matteson, the trustee of their father's estate for advice and assistance.

[7] On September 8, 1897, he married Natalie Bayard Dresser (1869–1950) at Trinity Church in Newport, Rhode Island.

Natalie was the daughter of Brevet Major George Warren Dresser (1837–1883), a West Point graduate and an officer in the Union Army during the American Civil War,[9] and Susan Fish Le Roy (1834–1883).

[17] In 1901, his widow donated the funds to build Emmanual Episcopal Church in Newport, RI in his memory.

She died at her home, Harbor Court (now the Newport station of the New York Yacht Club), in 1950.

[4] On February 1, 1890, Brown became a charter member of the Rhode Island Society of the Sons of the American Revolution.

This was because he never completed a formal membership application to join the Society despite having ancestors who supported the cause of American independence.