He married Mary Alma Jones on June 14, 1931, of this union was born two children, John Scott and Alice Ann Carter.

While working as assistant cashier, his mentor was former state senator Cortis K. Stacy, president of the bank and uncle by marriage.

The Carter home had become a popular meeting place for supporters of Bert Combs as they led a campaign against Harry Lee Waterfield's 1959 bid for Governor of Kentucky.

He held that position until 1964, when he assumed the role of Kentucky Auditor of Public Accounts in the Ned Breathitt administration.

He owned a 1,000-acre farm in Morgan County, Kentucky, where he grew tobacco, purebred Hereford cattle, and mined limestone.