Noted for his benevolence and paternalism, Warren built a commercial block adjacent to the mill which is named in his honor.

A son, Samuel D. Warren II, was an attorney who co-published an influential article entitled The Right to Privacy in 1890.

Daughter Cornelia Warren was an American farmer and an educational and social service philanthropist.

Son Henry Clarke Warren was an American scholar of Sanskrit and Pali.

Edward Perry Warren was an art collector and the author.