Ives was born on April 9, 1769, in Beverly in the Massachusetts Bay Colony in what was then British America, and was baptized on June 4, 1769.

[1] He also served as a trustee of Brown University, named in honor of his wife's family, for forty-three years, and in 1829, he presented the college with a philosophical apparatus.

[4] Through his eldest daughter, he was the grandfather of banker, industrialist, U.S. Army officer, state senator and philanthropist Robert Hale Ives Goddard.

[5] Through his son Moses,[5] he was the grandfather of U.S. Civil War Captain Thomas Poynton Ives (1834–1865) and Hope Brown Ives (1839–1909), who married Henry Grinnell Russell, one of the wealthiest men in Rhode Island.

[9] Between 1803 and 1806, Ives hired Caleb Ormsbee to build him a family home at 66 Power Street in the College Hill in Providence, Rhode Island.