FK Železničar was formed in 1924, when a group of workers from the Belgrade railway station began playing the game.

The club had many problems with their financial status but they continued to play.

The team began to compete in higher leagues.

In the early 1950s, they competed in the "Unified Sport League" (Jedinstvena sportska Liga).

In 1995 it becomes member of the Serbian League Belgrade, however its major success was achieved in 1999 when it gained promotion into the FR Yugoslav Second League – group North, where it stayed until 2003, when it was relegated by administrative decision because of the restructuring of the league competition system.