In the summer of 2004, they embarked upon the Horse the World Tour 2004, giving way to 85 shows in 90 days, spanning seven countries throughout North America and Europe.

With the new lineup of Engstrom, Isen, Winneke, Green, and Arkenstone, the band recorded and released their second album The Mechanical Hand (2005).

The band stated on their Myspace that "We left that tour because we ate really, really, really good pizza in Lou Malnati's in Chicago with Dave's grandparents.

The pizza was such that we were inspired by God to write music of the kind not heard in this world since Mozart was fed his first currywurst," though astute fans noticed that Eli Green was no longer in the band, having been replaced by Chris Prophet.

Drummer Jon Karel, having been hired as a substitute, returned to his activities in The Number 12 Looks Like You, leaving only Engstrom, Isen and Winneke.

[12] The band's most recent full-length album, Desperate Living, was released on October 6, 2009, with new drummer Daniel Pouliot and bassist Brian Grover.

In March 2017, HORSE the Band toured with Infinity Shred and Graf Orlock, closing out their sets with their first new song in eight years.

The EP, also referred to by fans as "A Reason to Live", was to be crowdfunded by Ethereum smart contract, which would disburse Ether cryptocurrency to the band members as payment for making progress on the album.

It is the band's first new release in over a decade and contains the two songs played on tour, "A Reason to Live" and "NoGimbus", as well as a cover of Nine Inch Nails' "March of the Pigs".