Mills first learned guitar before picking up drums when he was ten years old.
He played in the swing groups of Charlie Barnet and Boyd Raeburn in the 1940s, then with Jazz at the Philharmonic, Gene Norman, Babe Russin, Mannie Klein, Dizzy Gillespie, Benny Goodman, Rene Touzet, Sonny Criss, Andre Previn, Lionel Hampton, Stan Getz, Woody Herman, and Red Norvo.
Later in the 1940s he became interested in bebop and began playing in a style influenced by Max Roach.
Mills also recorded as a session musician in the 50s, working with artists such as Gerry Wiggins and Anita O'Day.
Mills owned and operated Larrabee through the mid-1980s, after which the studio was acquired by his son Kevin.