Derek Pressnall and Nick White, natives of Dunwoody, Georgia were drawn to the music scene in Omaha, Nebraska and moved.

The group released a limited-pressing eponymous 7" on Rue Royale Records, followed by their debut LP, Wild Like Children in 2004.

Wild Like Children garnered the band a substantial amount of critical acclaim and propelled them to national tours with Bright Eyes, Rilo Kiley, and Of Montreal.

Tilly and the Wall appeared on the Late Show with David Letterman on October 27, 2006, to perform "Bad Education".

The band toured with Jenny Lewis in support of her album featuring The Watson Twins, Rabbit Fur Coat.

They then went on their first headlining tour to promote Bottoms of Barrels, with acts such as David Dondero, Now It's Overhead, and Wolf Colonel supporting them.

Band members Jamie Williams and Derek Pressnall were married shortly after the album's release, on August 12, 2006.