Abdul Rahman Khan was born in a Khan family on 22 September 1926 to Yousuf Khan and Inayat Begum in Parbhani town of Hyderabad state of British India (now in Parbhani, Maharashtra, India), he was eldest amongst five children of his parents, having three sisters and two brother, he obtained his Bachelor of Arts and LLB degrees from Osmania University of Hyderabad, In 1955 he married Rasheeda begum from Hyderabad who was a school teacher by profession, they have 3 sons and 3 daughters.

his eldest son Irfan ur Rahman Khan is continuing his political legacy and is former Secretary of Maharashtra Pradesh Congress Committee from Parbhani.

who contested vidhan sabha elections from Parbhani in 2014 on Indian National Congress's ticket.

[1] He was known for his clean image and as an Indira Gandhi confidant in the political spectrum, he started his political career as a member of Parbhani municipal council and later became President of Parbhani municipal council and Marathwada Wakf Board.

Abdul Rahman Khan died due to cancer on 26 July 2007 in a government hospital of Parbhani at the age of 81 years.