In order to continue to play, many of the remaining footballers joined Berliner FC Favorit 1896.

Through the 20s and into the early 30s VfB Hermsdorf played second division football for the most part with only a single season appearance 1932–33 in the upper tier Oberliga Berlin.

Following the outbreak of World War II it became increasingly difficult to field a side and the club suspended play in 1943.

After a number of seasons in lower division play Hermsdorf climbed to the Amateurliga Berlin (II) in 1955 where they had an unremarkable eight-year run.

VfB Hermsdorf made a single season re-appearance in the Amateurliga in 1970–71 before falling to lower level local competition.