Rahmatullah played for the India national team and spent most of his career with Calcutta Football League giant Mohammedan Sporting.

[2][3] He later appeared with Bangladeshi outfit Dhaka Mohammedan and became one of the earliest Indians in post-independence period to take the international transfer, while other being legendary Mohammed Salim, who played for Celtic.

[10] Rahmatullah also represented Balaidas Chatterjee managed Bengal football team at the Santosh Trophy and won the tournament thrice in 1953–54, 1955–56, and 1958–59 seasons.

[11] Rahmatullah made his senior international debut for India against Burma on 26 May 1958 in the 1958 Asian Games, that ended up a 3–2 win in favour of them.

[19] In national team, Rahmatullah's teammates under coach Syed Abdul Rahim,[20][21][22][23] were like:[24][25][26][27][28][29] Ahmed Hussain, Peter Thangaraj, Nikhil Nandy, Samar Banerjee,[30] P. K. Banerjee, Kesto Pal, Neville Stephen D'Souza, Tulsidas Balaram, Sayed Khwaja Aziz-ud-Din,[31] Abdul Latif, Mariappa Kempiah, Chuni Goswami, Kannan.