He played college football for the Clemson Tigers, where he won the John Mackey Award as a junior, and was selected by the Indianapolis Colts in the third round of the 2012 NFL draft.

He started all 13 games in 2010 and was a second-team All-ACC selection after recording 33 receptions for 373 yards with a touchdown.

[11] Allen battled a knee injury late in the 2014 season, but started 13 games for the Colts and finished with 29 receptions, 395 yards, and 8 touchdowns.

[11] On November 8, 2015, Allen was poked in the eye by Denver Broncos' cornerback Aqib Talib.

[22] On November 12, on Sunday Night Football, Allen made his first reception with the Patriots when he caught an 11-yard touchdown pass from Tom Brady.

[23] Allen reached Super Bowl LII with the Patriots, but lost 41–33 to the Philadelphia Eagles.

[24] Allen was again not a part of the Patriots offense in 2018, as he only had three catches for 27 yards during the season, although he was praised by the team for his blocking.

Allen did win Super Bowl LIII when the Patriots defeated the Los Angeles Rams 13–3.