They had three daughters, Mary Rebecca and the twins, Isbell Stuart Turnbull and Jacqueline Segar Epes, all of whom outlived their parents.
For 35 years Gravatt was also a part-time trial judge in Nottoway County, after his election to the position by the Virginia General Assembly.
He handily defeated Wyatt T. Walker, whom Gravatt referred to as a "negro of the City of Petersburg", and who later became Executive Secretary of the Southern Christian Leadership Conference.
[2] The convention was necessary to amend the state Constitution to allow tuition grants to private schools, called segregation academies because they primarily served white parents who wanted their children educated in an environment isolated from non-white students, including African Americans.
On July 23, 1956, Gravatt gave a speech advocating continued segregation before the Defenders of State Sovereignty and Individual Liberties, with which he was heavily involved.
[6] From at least early 1955, Gravatt represented the Prince Edward County Commonwealth's Attorney or Board of Supervisors in several cases.
When he attempted to highlight the new integrated county school built using federal grants (after severe disturbances in August 1963),[7] as well as argue that schools were solely a matter of state jurisdiction, Justice Hugo Black chided Gravatt for evasiveness and Chief Justice Earl Warren tartly questioned him as to whether that meant black children had a "freedom to go through life without an education.