During the 2003–04 NBA season, however, O'Brien consistently fought with Celtics' general manager Danny Ainge over short-term versus long-term goals.

Ainge was looking to completely redo the roster, and traded Eric Williams and Tony Battie, two of O'Brien's favorite hardworking players in December 2003.

The 76ers made the playoffs in his one season as coach after missing the postseason the previous year, and although O'Brien had a multiyear contract, he was fired.

[4] He replaced Rick Carlisle, who was fired after four years, when the team failed in 2006–07 to make the playoffs for the first time in a decade.

On January 31, 2011, Pacers President of Basketball Operations Larry Bird relieved O'Brien of his coaching duties.

He was replaced by assistant Frank Vogel on an interim basis, who remained as the Pacers' head coach until the 2015–16 season.