[2] During his senior year of high school, Grimes averaged 29.5 points, 8.6 rebounds, 4.9 assists, 1.8 steals, and 1.5 blocks per game, and led the College Park Cavaliers to a 21–13 overall record.

On November 15, 2017, he committed to playing college basketball at the University of Kansas, choosing the Jayhawks over offers from Kentucky, Marquette, Texas, and eight other schools.

[16] As a junior, Grimes averaged 17.8 points, 2.0 assists, and 5.7 rebounds per game, sharing AAC Player of the Year honors with Tyson Etienne.

[19] In his first career start, Grimes scored 27 points and made seven three-pointers along with three rebounds and three assists in a loss to the Milwaukee Bucks.

[23] On February 8, 2024, Grimes, Ryan Arcidiacono, Malachi Flynn, Evan Fournier, and two second-round picks were traded to the Detroit Pistons in exchange for Bojan Bogdanović and Alec Burks.

[24] On July 6, 2024, Grimes was traded to the Dallas Mavericks in exchange for Tim Hardaway Jr. and three future second-round picks.

[25] On February 4, 2025, Grimes was traded to the Philadelphia 76ers alongside a 2025 second-round pick in exchange for Caleb Martin.

[29] Off the court, Grimes has volunteered with the Houston-based nonprofit, Play With Purpose, which works with at-risk young people in the area.