He played college football for the UConn Huskies, becoming the school's all-time leading rusher and earning first-team All-American honors in 2008.

Brown entered his redshirt freshman season in 2006 as a kickoff returner and second string running back behind senior captain and incumbent starter, Terry Caulley.

However, when Caulley went down with an injury before Connecticut's game against Rutgers on October 29, Brown was asked to make his first career start at running back.

Brown had his best outing of his freshman campaign in the next game when he rushed for 205 yards on 43 carries with 3 touchdowns (1 receiving) in Connecticut's 46–45 double overtime win over Pittsburgh.

He finished the year with 896 yards and 9 touchdowns (2 receiving), which was good enough to award him with a 2nd team All Big East nomination.

[2] Brown began his sophomore season with heightened expectations brought on by his very successful rookie campaign.

Brown rushed for 150 yards in the eighth game of the season at home in a 40–16 Connecticut victory over eventual Big East champion Cincinnati.

A 49-yard touchdown run in the second quarter would allow Brown to surpass Caulley as Connecticut's all-time leading rusher.

[6] On January 3, 2009, Brown was named Player of the Game in UConn's 38–20 victory over the University at Buffalo in the 2009 International Bowl in Toronto.

[9] Brown was considered one of the top three running back prospects available in the draft (alongside Knowshon Moreno and Chris Wells) and drew comparisons to Willie Parker.

[15] In his first action as a member of the team, Brown ran for 58 yards on five carries against the Minnesota Vikings in the first week of the preseason.

He had his best game as a professional in a Week 15 27–13 win over the Tennessee Titans when he rushed 16 times for 161 yards and scored a touchdown on an 80-yard run in the fourth quarter to seal the first victory of the season for Indianapolis.

Brown's 2012 season was hampered by some nagging injuries and the showing of rookie back Vick Ballard.