It was written by lead vocalist and keyboardist Victoria Legrand and guitarist Alex Scally and produced by them along with Chris Coady.

[2] The B-side is a four-track demo version of "Apple Orchard", a song which featured on the band's self-titled debut, recorded three years to the month before "Used to Be".

"[7] The 405 said the song "builds to a quiet crescendo of noise that is capped with some heartbreaking and mezmerising vocals," while also including it on the best "track to download" off the album along with "Norway.

"[8] musicOMH said the track is given "extra dimensions with sonic architecture, intricate guitar picking, and Victoria Legrand’s surprisingly meaty vocals.

"[9] Now Magazine says the songs effectively places "lyrical peaks into the slow-tempo slurry of basic backbeats and electronic clouds," while labeling it as the top track off the album.