Groh was also the head coach for the New York Jets of the National Football League (NFL) for one season, in 2000, tallying a mark of 9–7.

Throughout his career, Groh has been a friend and protégé of Bill Parcells, working with him for over 13 years in both collegiate and professional positions.

In 1987, Groh made his NFL debut as the special teams and tight ends coach of the Atlanta Falcons.

After a brief return to the college ranks in 1988 as the offensive coordinator for South Carolina, Groh joined the New York Giants coaching staff.

The 2001 Cavalier team posted a 5–7 record, and Groh, a native of New York City and Long Island, was involved in controversy surrounding comments regarding the September 11, 2001 attacks in his hometown.

While Groh acknowledged that the year would be a rebuilding experience and a slow start upset some fans, Virginia athletic director Craig Littlepage stated that, "Al will be our head coach" through at least the 2007 season.

[10] Littlepage, however, later refused to exercise an option to extend Groh's contract by one year stating that Virginia's expectations were higher than 5–7.

In the 2007 Virginia Cavaliers preseason, Groh was listed among the five worst coaches in college football by Sports Illustrated columnist Stewart Mandel.

[13] After leading the 2008 Virginia Cavaliers to a 1–3 start, including a 31–3 loss to a Duke team who had not won an ACC game in its previous 25 attempts, Groh was once again under pressure to resign.

[14] However, Virginia quickly turned the season around, reeling off 4 straight wins with the emergence of Marc Verica and the efforts of Cedric Peerman.

[15] Groh and his 2009 Virginia Cavaliers team opened the 2009 season with a shocking home loss to William & Mary of the FCS (formerly I-AA).

Groh's decision to field three quarterbacks in a spread offense for the game led to a record-tying seven turnovers for the Cavaliers.

The embarrassment of the loss coupled with fears of another losing season once again rekindled calls for Groh's ouster as Virginia head coach from many fans and analysts.

On January 15, 2010, it was announced that Groh would be taking the position of defensive coordinator at Georgia Tech for the 2010 season,[19] and coached there through the Yellow Jackets' loss to Clemson on October 6, 2012.

[3][20] While Groh was at Virginia, 13 Cavaliers were selected in the NFL Draft, while 19 others signed pro contracts as free agents.

During his first five years, Groh maintained a strategy of hiring young, ambitious assistants, and he hoped to build a network of protégés through the football ranks.

Mike London was an assistant with the Houston Texans before returning to Groh's staff and then taking the Richmond job.