[3] Alexander biographer Richard Younger describes the song as "a classic example of sixties pop songwriting that captures the thrill of first love.

"[4] Billboard Magazine described the song as "an emotion-packed theme with solid female chorus work," and also praised the feeling Alexander imbues his vocal with.

"[4] Producer Noel Ball claimed to have taken Alexander "way uptown" with the recording, creating what Younger describes as "a thrilling production that builds from a stark bass and drum into a mini-symphony with strings reminiscent of The Drifters' "There Goes My Baby.

[4] Authors David Hatch and Stephen Millward see a resemblance between "Where Have You Been (All My Life)" and early songs written by John Lennon and Paul McCartney.

[7][8] Music critic Richie Unterberger described the song as "cool" and "moody" but complained that the sound on this recording was too "muddy" to be able to figure out the lyrics.