During the Second Sino-Japanese War most warlords in China began to nominally unite against the Japanese invaders and Tang became an important member of Chiang Kai-shek's national defense committee.

Some writers pointed out that it was Tang who volunteered to serve as the commander of the Nanjing garrison and promised to fight until his death without any pressure from Chiang Kai-Shek.

He also placed the 35th and 72nd divisions at the port to prevent people from fleeing Nanjing, as instructed by Chiang Kai-shek's general headquarters at Wuhan.

The defense plan fell apart from the very beginning because the defenders were overwhelmed by Chinese troops fleeing from battles in the area surrounding Nanjing.

To carry out Chiang's directive, Tang would have had to have the Nanjing Garrison wage battle against the fleeing Nationalist troops before facing the Japanese assault on the city.

However, Chiang was extremely reluctant to give up the capital without a fight, and nobody else would dare to make such a decision and face the angry Chinese public.

Once the news reached Tang's headquarters that several units had abandoned their positions and fled against orders, it became obvious that a general retreat was inevitable.

The problem was that whoever gave the order to retreat would be blamed for losing the capital and face a very angry Chinese public, Tang was very reluctant to take the responsibility and the consequent blame alone and so he called a meeting that included every divisional commander and those of higher rank and showed them Chiang Kai-shek's permission to retreat when needed, a decision to be made by Tang's headquarters.

On 12 December, after two days of defending against an enemy with overwhelming numerical superiority, which was shelling the city with heavy artillery fire and aerial bombardment, and with many of his troops in open flight, Tang ordered a general retreat.

Frank Tillman Durdin of the New York Times and Archibald Steele of the Chicago Daily News wrote of the Chinese troops looting shops for food and removing their uniforms for civilian clothing, "Streets became covered with guns, grenades, swords, knapsacks, coats, shoes, and helmets.

After World War II he was not noticed until the fall of the Kuomintang regime, when Bai Chongxi asked Tang to go with him as the Nationalist force withdrew further south.

Tang refused to flee China, disguising himself and hiding in different places to avoid being captured by the Nationalist forces, breaking one of his legs in the process.