A member of the Democratic Party, Crisp was elected as a congressman from Georgia in 1882, and served until his death in 1896.

At the outbreak of the American Civil War, he was temporarily residing in Luray, Virginia, with his parents, who were in the middle of a Shakespearean play tour.

He enlisted in a local unit, the "Page Volunteers" of Company K, 10th Virginia Infantry, and was commissioned lieutenant.

He was appointed solicitor general of the southwestern judicial circuit in 1872 and reappointed in 1873 for a term of four years.

Crisp courted Clara Bell Burton, born in Ellaville, a small town in southwest Georgia, of wealthy and religious parentage.

One bright Sunday morning, when she was visiting her brother on the outskirts of Ellaville, Crisp drove Clara Bell in his buggy to his boarding place, and there in the presence of a few friends in the parlor, they were married.

He had been nominated for United States Senator in the Georgia primary of 1896, but he died in Atlanta on October 23, 1896, and was buried in Oak Grove Cemetery in his hometown of Americus.

The portrait was removed from public display in the Speaker's Lobby outside the House Chamber after an order issued by the Speaker of the House, Nancy Pelosi on June 18, 2020, due to Crisp having fought in the Confederate States Army.