The A to Z Recordings originate from a series of acoustic concerts by Australian rock musician, Paul Kelly starting in December 2004 with 100 songs performed alphabetically over four nights at The Famous Spiegeltent in Melbourne.

According to Rolling Stone's Jason Cohen, Kelly "might be creating the world's longest CD liner notes" at 568 pages.

[10] The boxed set was issued on 24 September 2010 by Gawd Aggie Records in Australia and Universal Import in North America.

[12] In January 2011 The Sydney Morning Herald's Bernard Zuel caught the first two nights of a four night set, he disputed the "accepted wisdom that acoustic performances were the test of a song's lasting quality" and preferred when Kelly's nephew Dan "add[ed] colour guitar or extra vocals to particular songs" or when Prior "provided scattered 'furnishings', playing clarinet, briefly bringing some operatic vocals to the party and noticeably brightening proceedings".

[13] Zuel felt the first night was "lacking some energy, vocal and physical" and that "the show's highlights came in the purely solo songs about or from the perspective of women".

[10] In November 2011, The A to Z Recordings were featured in two parts on The Weekend Planet program on Radio National (Australian Broadcasting Corporation) with Doug Spencer presenting tracks "beside another artist's utterly different (often, wordless) take on the same subject, or closely related theme".

[14] Spencer described some of the collection's tracks: "Adelaide" is a "21st century 'live' version of a song Paul wrote early in the 1980s, when his former home-city was a fresher 'wound'".

[14] Kelly's inspirations are seeing Dallas from the front of a tour bus in 1987 and reading Percy Bysshe Shelley's sonnet Ozymandias.

[15] While "You Can Put Your Shoes Under My Bed" addresses "a somewhat errant longtime friend/ former flame" with Kelly providing vocals, piano and harmonica.

It adopts the personae of two of the (many) indigenous Australians who were irradiated when Britain dropped atomic bombs into South Australia's desert country in 1957 and 1958".

[15] "Smoke Under the Bridge" tells the story of its protagonist, 'Banjo' Clark, "a young black man in mid-century rural Australia ... [who, later] was well-known, widely respected and much loved".