By March 1934, Nelson had joined John Dillinger's gang although Chase did not participate in their first holdup that month in Sioux Falls, South Dakota.

His relatively minor status within the gang was possibly the reason he was not present at the shootout with the FBI when federal agents raided the Little Bohemia Lodge near Manitowish Waters, Wisconsin on April 22.

[1] In the aftermath of the Mason City robbery, Nelson and John Paul Chase fled west to Reno, where their old bosses Bill Graham and Jim McKay were fighting a federal mail fraud case.

This was a disappointing amount considering the gang's past bank heists and, to make matters worse, a local police officer [4] was killed during their getaway.

Chase was not identified in the gunfight, authorities and journalists speculating either Alvin Karpis or John "Red" Hamilton as the second gunman, and decided to leave town while he had the opportunity.

On December 27, Chase was eventually arrested by police at Mount Shasta, California while working at a state fish hatchery and extradited to Chicago.

Chase had earned the ire of FBI Director J. Edgar Hoover, presumably for his involvement in the murder of Cowley and Hollis, and once referred to him as "a rat with a patriotic sounding name".

In a memorandum to his field agents, Hoover wrote "Watch closely and endeavor to thwart the efforts of this priest who should be attending to his own business instead of trying to turn loose on society such mad dogs".

[1] Chase remained in prison for another decade before he was finally released on parole, despite Hoover's protests, on October 31, 1966, after 32 years, and moved back to the Bay Area.

[2] Chase worked as a custodian at St. Joseph’s Seminary Los Altos, California until his death from cancer on October 5, 1973, outliving Hoover by one year, five months, and three days.