Originally a supporter and close associate of Chiang Kai-shek, Zhang belonged to the left wing of the Kuomintang, advocating policies such as collaboration with the Communists against Japan and nationalization of foreign-owned businesses.

He then served in the local warlord armies of Yunnan and Guangxi before heeding the call of Nationalist leader Sun Yat-sen and moving to Guangzhou to become an instructor at the Whampoa Military Academy.

He participated in the Northern Expedition, and after the Nationalists established the new republican government in Nanjing, became the Commandant of the Central Military Academy in 1929, a post he held for 8 years, until 1937.

He joined in Generalissimo Chiang Kai-shek's successful campaign against Yan Xishan and Feng Yuxiang, and led the 5th Army in the 1932 battle at Shanghai against Japan, and participated in setting up air-defense at the capital of Nanjing, including Jurong airbase.

In 1940, he became the Director of the Political Department of the Military Affairs Commission, another highly sensitive position, indicating Chiang's continued trust in him.