The finished recording employs a complex vocal arrangement devised by Jerry Yester, which later inspired Brian Wilson of the Beach Boys in composing his 1966 song "God Only Knows".

Steve Boone began the earliest elements of "You Didn't Have to Be So Nice" at the Greenwich Village home of the parents of Joe Butler's girlfriend, Leslie Vega.

[1][2] Drawing inspiration from a remark he made on a date with Nurit Wilde, Boone started the composition on the piano as a basic melodic figure, which he initially titled "You Didn't Have to Be So Nice, I Would Have Liked You Anyway".

[9][10][11] The author Richie Unterberger writes that like many folk-rock acts, the Spoonful's style bent towards pop music,[12][13] and he considers "You Didn't Have to Be So Nice" "one of their poppier offerings".

He identifies several hooks within the song, including an accompaniment figure of stepwise descending triplets played on an electric piano, an instrument the Beatles employed heavily in 1965 and 1966.

[19][nb 3] Amid their busy TV- and live-date schedule,[23] the Spoonful recorded "You Didn't Have to Be So Nice" in November 1965 at Bell Sound Studios in New York City.

[6] The finished recording features similar elements to the band's debut single, "Do You Believe in Magic", including a drum fill introduction, a shuffling tempo and Sebastian playing the autoharp.