Among the friends and associates of his student days was Sanford E. Church, afterwards Chief Justice of the Court of Appeals of New York.
With little business to be transacted, Graves "lived the life of a country lawyer", with a practice "necessarily limited to small affairs and the details of office work".
At the end of 1857, Michigan restructured its judiciary so that the Supreme Court was separately elected, and Graves continued serving as a circuit judge until he resigned due to poor health, in 1866.
Then sixty-six years old, he reasoned that "the weakness of age might unconsciously come upon him and impair his ability to discharge the duties of the position".
[1] His granddaughter, Betsy Graves Reyneau, was a distinguished painter; although discouraged by her father from entering that profession on the grounds that it was inappropriate for a woman, she broke ties with the family to pursue her career.