In March 1926, Sơn was introduced by Li Fuchun, Cai Chang, Lin Biao, Liu Zhidan and others at the Whampoa Military Academy.

After graduating in October 1926, Sơn followed Ho Chi Minh's instructions and continued to work at the Whampoa Military Academy, and joined the Kuomintang.

On 12 April 1927, Chiang Kai-shek carried out a purge of thousands of suspected Communists and dissidents in Shanghai, and began large-scale massacres across the country collectively known as the "White Terror".

With the consent of Ye Jianying and Nie Rongzhen, Sơn and Ho Chi Minh went to Thailand in order to organize overseas Vietnamese to join the revolution.

Due to his rich practical experience and theoretical background, he was assigned to teach at the newly established Central Military Political School of the Red Army in Ruijin, Jiangxi Province.

In January 1934, at the Second National Congress of Delegates of the Chinese Soviet Republic, Sơn was elected a member of the Central Committee of the CCP and as an ethnic minority representative.

In July 1937, at the beginning of the Second Sino-Japanese War, he followed Commander-in-Chief of PLA Zhu De and the 115th Division across the Yellow River to Shanxi Province to establish an anti-Japanese base at the Wutai Mountains.