Born in Levski, Pleven Province, Milanov began playing football at the local club, before joined the Litex Lovech Academy, together with his twin brother Georgi, in 2005.

[3] On 3 December 2009, Milanov made his Litex first-team debut in a 4–0 away win over Pirin Gotse Delchev in the Bulgarian Cup, coming on as a substitute in the 72nd minute.

[4] He made his league debut in a 4–2 away win against Sportist Svoge at Chavdar Tsvetkov Stadium on 16 May 2010, in the final game of the 2009–10 A PFG season, playing the full 90 minutes.

During the 2011–12 season, Milanov formed a partnership with Nikolay Bodurov in the centre of defence and has become an established first team player.

[10] He made his debut for the team on 1 March 2017 after coming on as a substitute and scoring the equalizer for the 1:1 draw against Ludogorets Razgrad.