A former member of the People's Action Party (PAP), he served as the Member of Parliament (MP) for Nee Soon Constituency from 1980 to 1988 and the MP for Nee Soon South Single Member Constituency from 1988 to 1996, both of which he served two terms, until they were absorbed by Ang Mo Kio Group Representation Constituency.

He also served as the first dean of the Science Faculty for the National University of Singapore (NUS) from 1980 to 1996.

[10] In the 1984 Singaporean general election, Koh remained as the MP representing Nee Soon Constituency after winning 74.24% of the vote against Singapore United Front's (SUF) Quek Teow Chuan's 25.76% of the vote.

[11][12] During the 1988 Singaporean general election, Koh became the MP representing Nee Soon South Single Member Constituency after winning 64.9% of the votes against Singapore Democratic Party's (SDP) Yong Chu Leong with 30.7% and United People's Front's (UPF) Munjeet Singh with 4.39%.

[13] In the 1991 Singaporean general election, Koh secured a second term as the MP representing the Nee Soon South Single Member Constituency by winning 52.9% of the vote against SDP's Low Yong Nguan.