Before his transfer to Bayern Munich, he played 1956–57 in the Oberliga Nord with Arminia Hannover and after the relegation of the club from 1957 to 1960 in the second tier, which in this part of Northern Germany was the Amateuroberliga Niedersachsen.

After the promotion to the Bundesliga in 1965 until 1970, Olk served as Bayern's captain, a function in which he was succeeded by Franz Beckenbauer.

He stayed there until 1973, although he had been relieved of his manager duties by December 1972, and was replaced in this role by Jiří "Georges" Sobotka.

[5][6] From 1995 to 1997, he was in charge of El Zamalek of Cairo, Egypt, with which he won the African Cup of Champions Clubs of 1996, where they prevailed in the finals over Nigeria's Shooting Stars SC.

Between 1986 and 1988, he served under Udo Lattek and Jupp Heynckes, being part of the team that won the German championship of 1987.