He was the Congress candidate for the Varanasi parliament seat in the 2014, 2019, and 2024 general elections, and lost to Narendra Modi.

[8][9] He became an associate of Brijesh Singh, after his elder brother Awadhesh Rai was shot dead in Lahurabir area, allegedly by Mukhtar Ansari and his men in 1994.

In his FIR, Anil Singh stated that Ajay Rai and others had fired at his jeep in the Cantonment area on 20 August 1991.

He went on to retain the seat in the 2002 and 2007 elections, defeating Awadhesh Singh of Bahujan Samaj Party (BSP) by much larger margins.

[14] However, in 2009, he resigned as an MLA, after BJP refused to give him a ticket for contesting the Lok Sabha elections from Varanasi.

His supporters allegedly campaigned for Joshi to ensure the defeat of his arch-rival Mukhtar Ansari in a communally-polarized environment.

[19] In 2010, Rai staged a sit-in to protest the arrest of the local corporator Sanjay Singh and his associate Jitin Jha.

[2] On 5 October 2015, Rai was arrested for his alleged role in the violence and arson in Varanasi during a march by seers and other local leaders against police action on protesters opposing ban on immersion of Ganesha idols in the Ganga river.

In 2019, Rai lost the Lok Sabha Elections from Varanasi constituency as a Congress candidate against Narendra Modi, securing third place.

[25] In 2024, Rai lost the Lok Sabha Elections from Varanasi constituency as a Congress candidate against Narendra Modi, securing second place.