The club was formed in 1921, and played its first competitive match on 17 September 1921, when it lost 5–1 against SAŠK Napredak.
As of 2024, Željezničar is one of only four clubs never to have been relegated from the top level of Bosnian football, the others being Sarajevo, Zrinjski Mostar and Široki Brijeg.
[1] Yugoslav League seasons 1946–1992 references:[2] Top goalscorer; Name and Goals in bold indicate overall top league scorer for the season.
Željezničar competed next in 1994–95 season of First League of Bosnia and Herzegovina (see table below).
In seasons 2000–01 and 2011–12 the club won a double; national cup and league titles.