Formed in Athens, Georgia, in 2001, they later moved to Omaha, Nebraska, and became part of the Saddle Creek Records music scene, which also included Bright Eyes, The Faint, and Cursive.

[2][3] They fronted a band called Little Red Rocket, which released two albums, Who Did You Pay (1997, Tim/Kerr) and It's in the Sound (2000, Monolyth Record Group), which earned them frequent comparisons to Veruca Salt.

[6] The pair signed to the Warm Electronic Recordings label and released their self-titled debut in 2001, produced by Eric Bachmann (Crooked Fingers and Archers of Loaf).

[13] After meeting Fink and Taylor in Athens, Bright Eyes frontman Conor Oberst invited the band on tour and introduced them to the people at Saddle Creek, the label to which he was signed.

Pitchfork called it a departure from their previous records, with "newly arresting manipulations of electronic elements interlock[ing] with the duo's traditional organic fare, as well as bolder and sharper production".

[19][20] Fink and Taylor reformed the band for a one-off show in Los Angeles at the iconic Troubadour on November 30, 2008, accompanied by Andy LeMaster of Now It's Overhead and Nick White of Tilly and the Wall.

[21] In 2009, Taylor announced on KCRW's Morning Becomes Eclectic that the duo had reformed on a semi-permanent basis, with the intention of playing "five or six" shows before working on a new Azure Ray album.

[35][36] On September 26, 2018, Stereogum announced the first new Azure Ray record in six years, an EP titled Waves, and premiered a new single, "Palindrome", praising it as a "mournful string-driven ballad which offers the best of the band's spectral harmonies, woodsy storytelling and aching wisdom".

Fink is also a member of Art in Manila, O+S, Harouki Zombi, Closeness, and High Up, and has played with or appeared as a guest on records by The Faint, Pete Yorn, Old Canes, and others.