Described by Pitchfork as "a born storyteller who's chosen rock as his medium,"[2] Finn has released five solo albums: Clear Heart Full Eyes (2012), Faith in the Future (2015), We All Want the Same Things (2017), I Need a New War (2019) and A Legacy of Rentals (2022).

[8] After moving to New York City, Finn got a job at a live music webcasting company called Digital Club Network.

[13] He worked with Minneapolis rapper P.O.S on the song "Safety In Speed (Heavy Metal)" which was released on the 2006 album Audition.

Always close to the Twin Cities music scene, he also contributed to Minnesota musician Mark Mallman's song "You're Never Alone in New York" on the 2009 album Invincible Criminal.

[14] In 2011, Finn performs lead vocals for a Minnesota Twins tribute song "Don't Call Them Twinkies" on The Baseball Project's second album Volume 2: High and Inside.

The EP was originally made available in June 2015 as a pre-order download via PledgeMusic, but was given a full release on March 4, 2016 via Partisan Records.

On April 4, 2025, Finn will release his sixth solo studio album, Always Been, produced by The War on Drugs frontman Adam Granduciel.

[20] Finn is most notable for his third-person narrative lyrical style, wherein he frequently makes reference to literature, pop culture, adolescence, partying, religion, and drugs.

[23] Finn's lyrics have been a frequent point of praise for The Hold Steady[24][25] with Uncut Magazine describing his style as "narratives driven less by the wordy exposition of yore than acute observation, devastating detail, by turns exclamatory, epigrammatic and grainily authentic.

"[26] In a review of A Legacy of Rentals (2022), veteran critic Robert Christgau describes Finn's albums as an "ongoing series of musical short stories" in which "not everybody loses... but for sure nobody wins", noting the preponderance of protagonists who are very likely nonvoting "all-white casualties of finance capital and the fossil fuel cartel".

[27] Finn has indicated that some of his greatest lyrical influences include Blake Schwarzenbach from Jets to Brazil and Jawbreaker as well as Bruce Springsteen.

In an interview with GQ, Finn discussed his love for Danko: "People roll their eyes about his solo records, but I'm just happy to have more songs that I can listen to him sing; his voice is incredible...