[3] His paternal grandfather, George W. Patterson of Buckingham County, Virginia was a second lieutenant in the Confederate States Army during the American Civil War.

[5] Patterson attended the public John Marshall High School in downtown Richmond, serving as first captain of its corps of cadets during his senior year.

[6] After graduating high school in 1944, he entered the Virginia Military Institute and was elected vice president of his class but was expelled the following year for being a troublemaker.

He was a member of the school's football, polo, and track teams as well as its honor court and an assistant editor of the student newspaper, The VMI Cadet.

[6] The 1949 Bomb yearbook highlights his work on behalf of the school's "States' Rights Club," an organization that was formed to support Strom Thurmond's segregationist 1948 presidential campaign.

[11] Patterson later married Anne Marie Whittemore, a partner at McGuireWoods who served as his co-counsel in the VMI case, on November 15, 2003.