He played college football for the Houston Cougars, and was selected by the Bills in the first round of the 2019 NFL draft.

In Oliver's senior season, Westfield went 11–1 into the 2015 UIL playoffs, where they lost 28–21 in the Area final to later state runner-up Austin Lake Travis.

[3][4] Receiving offers from a multitude of powerhouse schools, including Alabama, Florida, LSU, Oklahoma, Texas, and Texas A&M, Oliver chose to play college football at the University of Houston, becoming the first recruit rated by ESPN as a five-star to commit to a school outside the Power Five conferences.

[9] In 2017, Oliver's sophomore season, he was selected to the Walter Camp All-America first team and won the Outland Trophy, presented to the nation's top interior lineman.

[16] In week 7 against the Miami Dolphins, Oliver recorded his first career sack on Ryan Fitzpatrick in the 31–21 win.

[31] He made 16 starts and appearances in the 2023 season, contributing a career-high 9.5 sacks, 51 total tackles (34 solo), one interception, three passes defended, and one forced fumble.

[33] On May 16, 2020, Oliver was arrested in Houston on charges of driving while intoxicated and unlawfully carrying a weapon and held in Montgomery County Jail before posting bail the next day.