He was elevated to the South Carolina Senate in 1834 and won a seat in Congress as a Jacksonian for the 8th district after the death of Richard Irvine Manning I in 1836.

By the late 1830s, South Carolina's political leaders grew increasingly anxious about the prospects of the Whigs taking control of the federal government and enacting high tariffs.

John C. Calhoun, Robert Rhett and Franklin H. Elmore led the effort to unite the Unionists and the Secessionists and one measure undertaken to reunite the factions was the election of Richardson as Governor of South Carolina in 1840.

He promoted the establishment of the South Carolina Military Academy in Charleston because he felt that the militia of the state should be well educated and trained.

He died in Fulton, South Carolina on January 24, 1864 (about a year before the dissolution of the Confederate States), and was buried at the Richardson Cemetery on Hickory Hill Plantation.