The firm of Hill, Martin, & Robinson and the Virginia chapter of the NAACP, including Thurgood Marshall, served as Swanson's legal counsel.

[5] Chief Justice John J. Parker of the Fourth United States Circuit Court of Appeals, Federal District Judge John Paul, and Circuit Appeals Judge Morris A. Soper presided over the ensuing case, Swanson v. Rector & Visitors of Univ.

Despite facing a climate of racial prejudice at UVA, Swanson took initiative to be an active participant in University life beyond his graduate work.

He attended football games and was a season ticket holder to the University's Tuesday Evening Concert Group.

Swanson was a member of the University's integrated YMCA and helped organize that group's Committee for Racial Understanding.

[2][9] In 2016, Swanson's historic court victory was commemorated with the installation of a plaque at the former courthouse, now the current downtown location of the Jefferson-Madison Regional Library.