The former chief justice left Kentucky in part because of Clay's growing presence in the state.

[2] In addition, Nashville was growing rapidly as the chief city in the Middle District of Tennessee.

He was elected as a Jacksonian in 1829 to the United States Senate to fill the vacancy in the term ending March 4, 1833, caused by the resignation of John H. Eaton to join the Cabinet of President Andrew Jackson.

Reelected in 1832, Grundy served from October 19, 1829, to July 4, 1838, when he resigned to accept a Cabinet position.

Grundy was appointed as Attorney General of the United States by President Martin Van Buren in July 1838.

He resolved the question of whether he was eligible to be elected as Senator while holding the office of Attorney General by resigning on December 14, 1839.

He was reelected by the Tennessee legislature to the Senate the same day, serving from December 14, 1839, until his death in Nashville, a little over a year later.