He joined the Chinese Communist Party (CCP) in July 1947.
Since then, Cao had served as the Director of Research Office of Organization Department of CCP Central Committee (1978–83), the Deputy Head of Organization Department of CCP Central Committee (1983–87), and the Deputy Director of Research Office of Secretariat of CCP Central Committee (1987–88).
In 1988, he was elected the Executive Deputy Secretary-General of the 7th National People's Congress (NPC) Standing Committee.
In 1993, he was elected the Secretary General of the 8th NPC Standing Committee.
Later, he served as one of the Vice Chairmen of the 9th National People's Congress (1998–2003).