He made his name at Rapid in the next years to secure a move to Bundesliga side Werder Bremen where he would spend eight years,[2] divided in two periods by a season at Bayern Munich where he won the 1996 UEFA Cup, beating Girondins de Bordeaux in a two-legged final.

[3][4] After returning to Rapid in 2002, he decided to end his career with Major League Soccer franchise Los Angeles Galaxy in 2004.

Herzog made his debut for Austria in an April 1988 friendly match against Greece and was a participant at the 1990 and 1998 World Cups, scoring one goal in the latter tournament, a spot kick against Italy.

[11] On 1 August 2018, Herzog was announced as coach of Israel's national team, a move which was divisive due to his last minute strike to deny them a play off place in their 2002 World Cup qualifier.

[15] His father is Anton "Burli" Herzog (born 1941), who played in the Austrian Bundesliga between 1961 and 1975 for clubs like Austria Wien, Wiener Sport-Club and Admira/Wacker Mödling.