"Little Bother" came about as King Princess and the song's featured artist, Fousheé, shared direct messages with each other.

The piano line in "Crowbar" was contributed by Aaron Dessner, though it was originally shared to Gruska, as part of an unrelated project.

[16] The creation of "Change the Locks" was inspired by a slump in King Princess's relationship with creative producer Quinn Wilson, which was caused by the Coronavirus pandemic.

[23] The album's opening track, "I Hate Myself, I Want to Party" describes depression in a gritty manner and includes sinister-sounding production.

[20] "Change the Locks" also involves a guitar in its sound, while "Crowbar" makes use of heavy drums,[20] and finds King Princess comparing her breaking relationship to the titular object.

[20] Hold On Baby received generally positive reviews from music critics,[26] who praised certain attributes like its production, instrumentals,[27] vulnerability, and vocals.

[29] The Line of Best Fit reviewer Ims Taylor directed praise toward the album's instrumentals, and pointed out that the singer's level of confidence had increased on the record.

[30] El Hunt of NME was less positive, saying that the "best moments on Hold On Baby unearth a raw sense of introspection"; the reviewer criticized the record's cohesion and pacing.

[25] Charles Lyons-Burt, writing for Slant Magazine, criticized the album's melodies, remarking that "never quite come to life as convincingly as they should".

[32] Martyn Young, in his review of the album for MusicOMH, opined that the record makes a strong case for the artist's talents.

The reviewer had reservations about the restrained nature of Hold On Baby, saying that it makes parts of the album "[feel] a little disappointing".