His ancestors, a leading family of Stockbridge, Massachusetts, established a tradition of literary achievement, including authors Catherine Maria Sedgwick and Henry Dwight Sedgwick III.

He has been credited with discovering many writers and with having the Atlantic Monthly to be the first national magazine to publish a work of Ernest Hemingway's (the short story Fifty Grand, July 1927).

Mabel Sedgwick designed the gardens at Long Hill, the 114-acre home in Beverly, Massachusetts.

Sedgwick's son Ellery Jr grew to become a significant player in finance and investments, Cabot was a career diplomat with the US State Department and the father of actress and author Paulita Sedgwick, Theodora worked extensively in South-East Asia and was the wife of Brigadier General William Bond, and Henrietta became a well known horticulturalist in her own right.

He died in 1960 in Washington, D.C., and is buried in the Sedgwick family plot in Stockbridge.