In an interview with Rick Rubin on the podcast Broken Record, Kevin Parker said he wrote "Feels Like We Only Go Backwards" after being inspired by "Walk in the Park" by Beach House.

[3][4] The music video was directed by Joseph Pelling and Becky Sloan,[5][6] best known as creators of Don't Hug Me I'm Scared web series.

Pitchfork named it the 7th best song of 2012, and said of the track; "it will blow your brain back to its most purely joyous and least cynical recesses.

[10] The Irish sports podcast Second Captains used the song as the theme music for its 2014 FIFA World Cup coverage.

The song plays over the credits of the film Shimmer Lake, which features a narrative told in reverse.