Ali was born in a small town about twenty miles from Lucknow, the capital of Northwestern Provinces, during India's period under British rule.
He went to England in 1905 to join Lincoln's Inn in order to obtain a degree in law.
He was appointed inspector of Schools, but after some time he decided to resign the post and began practicing as a barrister in Gorakhpur.
During the Quit India Movement, he joined the Congress and very soon became a force to reckon with in the party.
He served on the executive committee of the Congress and was very popular in whole of the eastern region of India.