Cremins attended All Hallows High School in the Bronx, New York, where he was born to Irish immigrants from County Kerry.

Cremins, known as "Cakes",[citation needed] was also the captain of South Carolina's 1969–70 team which went 25–3 and won USC's first (and only) ACC regular season title.

At age 27, Cremins became one of the youngest NCAA Division I head coaches in history when he took charge of the basketball team at Appalachian State University.

Cremins's performance at Appalachian State gathered him some national attention in the NCAA coaching ranks, including catching the eye of Georgia Tech athletic director Homer Rice.

After Rice persuaded him to come to Atlanta, Cremins was hired as the Rambling Wreck's new head basketball coach at the close of the 1981 season, on April 14, 1981.

First there was Mark Price (the Cleveland Cavaliers) and John Salley (the Detroit Pistons) in the early 1980s, followed by Duane Ferrell, Tom Hammonds, Dennis Scott, Brian Oliver, Kenny Anderson, Jon Barry, Travis Best, Stephon Marbury, Jason Collier and Matt Harpring.

Cremins assisted former NBA coach Lenny Wilkens in the American basketball team's appearance in the Summer Olympic Games of 1996 in Atlanta.

On March 24, 1993, Cremins agreed to coach basketball at his alma mater, the University of South Carolina, before changing his mind and deciding three days later to continue at Georgia Tech.

Cremins also raised money for a five-to-six week summer program, half of which include disadvantaged children, the Hilton Head Basketball Camp 101.

Another highlight of Cremins' tenure was a home upset over a North Carolina team ranked number nine in the country on January 4, 2010.

[2] Having taken a medical leave of absence on January 27, 2012, which lasted for the duration of the 2011–2012 season, Cremins retired from coaching on March 19, 2012, citing physical exhaustion.