Jones began his career in 1973–74 with the ABA's San Diego Conquistadors, coached by Wilt Chamberlain.

In the 1974 ABA Western Division Tiebreaker Game, Jones helped San Diego to a 131–111 win over the Denver Rockets, posting a stat line of 16 points, 10 rebounds, 6 assists, and 6 blocks.

"We had so much talent on those 76ers teams that [Coach] Gene Shue said all he wanted his centers to do was play defense and rebound.

Free) (16.3 points per game) propelled the squad to a 50–32 regular season record and a 1977 NBA Finals meeting with the Portland Trail Blazers.

Jumping out to a two-game lead, the 76ers appeared to be headed for the title, but the Trail Blazers rallied for a dramatic 4–2 Finals win.

For the season, Jones averaged 6.0 points and 8.1 rebounds and finished sixth on the team in minutes played.

Philadelphia won the Atlantic Division again in 1977–78 but lost to the Washington Bullets in the Eastern Conference Finals.

In 1978–79 the 76ers slipped a bit, finishing second in the Atlantic Division to Washington and losing to the San Antonio Spurs in the conference semifinals.

That season, on February 18, 1979, Jones grabbed a career high 27 rebounds, along with scoring 19 points during a 110–107 win over the Milwaukee Bucks.

Although Philadelphia finished behind the Boston Celtics in the regular season, the 76ers tore through the playoffs, advancing past the Celtics in the Eastern Conference Finals where Jones led the 76ers to a Game 2 win with 22 points and 11 rebounds,[5] before coming up short against the Los Angeles Lakers in the Finals.

Jones played two seasons in Houston (joining his brother Major Jones on the Rockets' squad), one in Chicago, four in Portland, where he would play meaningful playoff minutes despite being in his late 30s, most notably at age 37 starting every game of Portland's first round loss to Houston during the 1987 NBA Playoffs while averaging 7.8 rebounds per game (second best on the Trail Blazers, only behind Steve Johnson),[6] and one final season in San Antonio.

When he retires, I think they should have a farewell tour for him", Larry Brown, Jones's coach with the San Antonio Spurs, told USA Today.

Oliver was the first of the Jones brothers to play basketball at Georgia's Albany State university and later became head coach for 28 years at the school.

Jones died of a heart attack while at a driving range near his home in Stockbridge, Georgia on September 21, 2014.