Before becoming the first successful player of Red Star Belgrade, Mitić started his career in 1937 on the football fields of Košutnjak.

The Second World War halted his football development, but he continued playing in 1944 for the engineer squadron under which he fought along with fellow future Yugoslavia national team players Predrag Đajić and Jovan Jezerkić.

One of his fondest memories was the first Yugoslav championship title, won in a dramatic finish, when the first place Dinamo Zagreb lost their lead by slipping in the last three rounds.

The disciplinary committee of the Football Association of Yugoslavia suspended all the players of Red Star (except Vladimir Beara and Kostić) for a month.

The club did not appeal against the decision, and Rajko Mitić made the most positive impression possible on the shocked football public.

On 29 November 1958, he officially retired from football at the age of 36, after Red Star's convincing Cup win of 4–0 against Velež.

He took part in two Olympic tournaments: London 1948 and Helsinki 1952,[3] where he captained the Yugoslavia squad that included many notable players of that era.

Before the match against Brazil at the Maracanã in Rio de Janeiroo, while he was entering the pitch from the tunnel, he lifted his head suddenly and hit a medal lid.

He was part of the coaching staff at Red Star from 1960 until 1966 when he became a member of the national team selection committee which consisted of Mitić and Aleksandar Tirnanić, Miljan Miljanić, Vujadin Boškov and Branko Stanković.

His biggest achievement as a coach was at the 1968 UEFA European Football Championship held in Italy, when the national team (led by Dragan Džajić) won the silver medal.