On August 20, 1931, Button married nurse Kathleen Mary Antoinette Cheape (1907–2007), who converted him to the Episcopal Church, and they had a son and a daughter.
During his fifteen years in the Senate, Button became associated with the organization's most conservative wing: backing fiscal conservatism and ultimately Massive Resistance (which originated to thwart the racial integration provisions of Brown v. Board of Education which the U.S. Supreme Court decided in 1954 and 1955).
The conservative slate ultimately elected included Button as attorney general, on a ticket with former attorney general Albertis Sydney Harrison for governor (Harrison resigning his position in order to run and being replaced by Frederick Thomas Gray until Button's election) and his close friend Mills E. Godwin Jr. as Lieutenant Governor.
During his two terms as attorney general (he was reelected in 1965 as Godwin won the governorship), Button defended racial segregation and criticized the U.S. Supreme Court as too liberal.
A heart attack Button suffered in 1967 may have convinced him not to seek re-election in 1969, but instead return to his private law practice in Culpeper.