Prior to forming Leaving Rouge, Hoen had fronted Thoughts of Ionesco, a Detroit-based noise-core band whom Alternative Press had referred to as "an ultimate expression of pain-through-sound."
According to a 2004 article in Real Detroit Weekly, Hoen claimed his change in musical direction drew skepticism from fans of Thoughts of Ionesco, and that the band had a difficult time finding an audience.
A departure from the more experimental tones of White Houses, Elsewhere received mixed reviews and the band broke up before the end of the year.
About the album, All Music Guide's Nick Raggett wrote, "As compared with the more stretched-out and reflective sound of the earlier version of the group, everything here is much more direct, aiming for play on a radio station that might not exist but probably should."
During its career, Leaving Rouge opened for Ted Leo, The Appleseed Cast, The Constantines, Kings of Convenience, Karate, and several other "indie" acts.