He was also a member of the United States House of Representatives for eight consecutive terms from Mississippi from 1885 to 1901.

He was the son of David M. Allen (1810–1875), a planter from Henry County, Virginia, and his wife, Sallie A.

Allen was admitted to the bar the same year and commenced practice in Tupelo, Mississippi.

He gained the nickname "Private John" Allen campaigning for Congress.

Allen declined to be a candidate for reelection in 1900 to the Fifty-seventh Congress and then he was appointed in March 1901 as United States commissioner to the St. Louis Exposition of 1904.