He held the post until April 1985 when he lost the seat to Wong Phin Chung of United Sabah Party (PBS).
This time, he won back his traditional stronghold of Kudat, and was appointed state minister of Tourism Development, Environment, Science and Technology.
When Musa Aman took over the helm of the state government in 2003, Chong was appointed Deputy Chief Minister as well as the person in charge of the tourism, culture, and environment portfolio.
It was then the rotation system was scrapped and from then onwards, the Chief Minister post was held by the United Malays National Organisation until the 2018 Sabah state election.
In April 2007, Chong resigned from the Deputy Chief Minister post under Musa Aman's cabinet, citing matters of principle.