He enrolled at nearby Ouachita Baptist University, in Arkadelphia, Arkansas in the fall of 1966, playing basketball for Coach Bill Vining.

In 1971–1972, Patterson averaged 25.3 points and 14.7 rebounds as Ouachita Baptist advanced to the second round of the 1972 NAIA men's basketball tournament.

The Bullets had received the draft pick spot earlier in the day in a trade with the Phoenix Suns for eventual Naismith Basketball Hall of Fame inductee Gus Johnson.

He played under Coach Gene Shue and alongside Hall of Fame inductees Wes Unseld and Elvin Hayes.

[6] Patterson played with the New York Nets of the American Basketball Association in the 1974-1975 pre-season under Coach kevin Loughery.