Jarvis began his career as a trainee with Norwich City, making history as the youngest player to appear in their reserve side when aged only 14.

[2] The younger brother of fellow Norwich academy product Ryan Jarvis, he progressed to become a trainee with the Canaries and made his senior debut as a second-half substitute for Dean Marney in a 2–1 League Cup defeat away to Birmingham City on 26 October 2005.

[6] After recovering, Jarvis played out the remainder of the season on loan at League One Rotherham United, making ten appearances.

[10] He found some familiar faces in the dressing room, as the Hatters' squad included ex-Norwich teammates Michael Spillane, Chris Martin and Paul McVeigh.

Jarvis played a total of 46 games for Luton in 2008–09, including in every round of the club's Football League Trophy success, scoring the winning goal in the quarter-final against Walsall.

[12][13] Though he was a regular player under Luton manager Mick Harford, his dismissal in October 2009, and subsequent replacement in Richard Money, saw Jarvis used sparsely.

[26] Whilst in non-League football with Luton, Jarvis was eligible to be called into the England C squad for friendlies against both Hungary under-23s and Poland under-23s in September and November 2009.