In 2004, the band toured Europe for the first time to support their second album, Beggars Would Ride.

Frank Vollmann relocated to Berlin in 2006, joining long-time comrades Fez Wrecker (Bonniwell Music Machine, The Fuzztones) and Benn Ra (Hatesex, ex-Diva Destruction), both of whom had coincidentally played in early incarnations of Frank the Baptist in San Diego in 1997–1998.

From 2006-2009, the band maintained a headquarters of sorts at a bar run by Frank and Fez, called "The Speakeasy Berlin".

In 2015, they self-released their 4th album As the Camp Burns as the long awaited follow up to the critically successful New Colossus.

2019 saw the group join Dark Dinensions label, and released their 5th album Road Omen.