[1] Withers served as head football coach at University of North Carolina at Chapel Hill during the 2011 season, at James Madison University in the 2014 and 2015 seasons,[2] and as the interim head coach at Temple following Stan Drayton's firing in 2024.

Withers started his coaching career at Austin Peay State University as the defensive coordinator in 1988.

Withers then moved to the National Football League (NFL), joining the New Orleans Saints as a defensive quality controller in 1994.

He was promoted to interim head coach from the defensive coordinator position after Butch Davis was fired on July 27, 2011.

From 2002 to 2004, the Titans grabbed 57 interceptions, the best ever for a three-year span for the franchise, fourth best in the AFC and eighth best in the NFL.

The Heels also ranked fourth in the ACC and 30th nationally in defense, even with injuries that made Withers play a different lineup every game.

Giovani Bernard became the first Tar Heel rusher in 27 years to rush for over 100 yards in four straight games.

Bernard extended his streak of 100 yard rushing games to five in UNC's 30–24 loss to University of Miami.

Dwight Jones's 79 receptions and Bryn Renner's 23 TD passes set single season records for the Tar Heels.

[7] On December 20, 2013, Withers accepted the position of head football coach at James Madison University.

[9] On January 6, 2016, Withers accepted the position of head football coach at Texas State University.

On February 26, 2019, it was reported that Withers was hired as the defensive backs coach of the New York Giants.

In January 2021, Withers was announced as the defensive coordinator at FIU, reuniting him with former North Carolina head coach, Butch Davis.