Rudy joined VfB Stuttgart's youth academy in 2003 and started his senior career in 2007 with the club's reserve team, playing in the semi-professional Regionalliga Süd.

[13] During the summer of 2008, he also signed a contract with VfB Stuttgart's first team, for which he made his competitive debut in the first round of the DFB-Pokal on 10 August 2008 in their 5–0 away victory over Hansa Lüneburg.

[6] On 28 August 2010, Rudy made his debut in a Bundesliga match as a substitute by replacing Peniel Mlapa in the 89th minute in a 1–0 victory over St.

[16][17] In his first Bundesliga match with the club, he provided an assist from the free-kick to his fellow Bayern newcomer and his former Hoffenheim teammate Niklas Süle in a 3–1 win over Bayer Leverkusen.

[10] Rudy debuted for the German senior team on 13 May 2014 in a friendly against Poland in Hamburg's Imtech Arena.

[24] On 4 June 2018, Rudy was named in Joachim Löw's final 23-man squad for the 2018 FIFA World Cup.