Patterson played college football for the Hutchinson Blue Dragons before transferring to the Tennessee Volunteers, where he earned first-team All-SEC honors.

[3] Patterson attended Northwestern High School in Rock Hill, South Carolina, where he played football, basketball, and ran track.

[5] As a senior in 2008, he caught 75 passes for 944 yards with 18 touchdowns, leading his team to the State Class 4A Division II championship game, and was named All-state selection.

He spent part of 2009 beginning a post-graduate year at North Carolina Tech Preparatory Christian Academy, but did not play football, and left prior to the spring to enroll at Hutchinson Community College.

Rated as a 5-star prospect by 247Sports.com and the top JUCO player in the country, Patterson received scholarship offers from upper-tier Division I programs.

[16] He ultimately chose to attend the University of Tennessee over LSU, Auburn, Georgia, Ole Miss, and others.

[21] After a limited role in the 47–26 victory over the Akron Zips, he had a 46-yard rushing touchdown in a 51–44 shootout loss to the rival Georgia Bulldogs on September 29.

[27][28] In 12 starts Patterson racked up a school-record 1,858 all-purpose yards (set by Reggie Cobb in 1987) to rank first in the Southeastern Conference, tied for 18th in NCAA.

[37][38][39] In the season-opener on September 8, Patterson caught his first NFL pass from quarterback Christian Ponder for a 10-yard gain and had two kicks returned for 54 yards.

[42] In Week 8, Patterson returned the opening kickoff 109 yards for a touchdown, tying the NFL record for the longest play ever.

After Patterson's performance against the Green Bay Packers, he was named NFC Special Teams Player of the Week.

[44] On December 1, Patterson ran a touchdown for 33 yards, helping the Vikings win in overtime over the Bears.

[45] Patterson's best receiving performance of the season came in Week 14 against the Baltimore Ravens with five catches for 141 yards; in that game, he scored his longest receiving touchdown when he took a bubble screen pass from Matt Cassel 79 yards to the end zone with just 45 seconds remaining in the fourth quarter.

[51] On January 3, 2014, Patterson was named as a starter on the 2013 Associated Press (AP) All-Pro team as the kick returner.

[52] Patterson suffered through a disappointing 2014 season under new head coach Mike Zimmer and offensive coordinator Norv Turner.

[54] Patterson ended up losing his starting job to Charles Johnson, a practice squad player the team picked up from the Cleveland Browns.

[55] Patterson had an opportunity to reacquire his starting position after veteran wide receiver Greg Jennings was released during the off-season.

[56] After newly signed veteran Mike Wallace established himself as the starter and rookie Stefon Diggs emerged as a premier receiver in Charles Johnson's absence, Patterson was dropped to fifth position and relegated to return duties.

[64] He joined a position unit that contained Michael Crabtree, Amari Cooper, Seth Roberts, and Johnny Holton.

Coming out of the tailback position, Patterson led the team in rushing, gaining 61 yards on 11 carries and scoring a touchdown.

[86] In November, Patterson recorded 294 kickoff return yards on ten attempts and four tackles as a gunner on punts.

Patterson's 29.9 kickoff return average was also the second-highest in Bears' franchise history behind Gale Sayers.

The touchdown also set the franchise record for the longest kickoff return in team history, surpassing Sayers' 103-yard score in 1967.

[96] In Week 4 against the Washington Football Team, Patterson recorded three receiving touchdowns, a career high.

[99] In Week 9 against the Saints, Patterson had six receptions for 126 yards, including a 64-yard catch at the end of the game that put the Falcons in range for a game-winning field goal.

[101] In the same game, Patterson also made his first appearance on defense for the Falcons, playing one down at safety during a hail mary attempt.

[114] Patterson appeared in 14 games in the 2023 season but fell to third on the running back depth chart behind rookie Bijan Robinson and Tyler Allgeier.

Though primarily used as a running-back, Patterson subbed in at wide receiver as well as the team's kick returner for much of the season.