Due to his family's close connections with former Chōshū Domain, he was officially registered as a resident of Yamaguchi Prefecture around that time.

He graduated from the 11th class of the Imperial Japanese Army Academy in 1899, and served as a junior officer in the Russo-Japanese War with the Guards 2nd Infantry Battalion.

In early November 1919, he succeeded in the hereditary title of hakushaku (count) under the kazoku peerage system upon the death of his father, and was raised in military rank to colonel.

In September 1926, the San'yō Main Line train he was riding on derailed in an accident that killed 34 people, but Terauchi was not injured.

He was the leading military commander in Osaka during the notorious "Go-Stop Incident", in which the verbal altercation between two young men—an off-duty soldier in uniform who had ignored a traffic light and a policeman, which developed into fistfights, and finally into a ministerial-level conflict between the Home Ministry and the Army.

[4] His inflammatory rhetoric brought about the collapse of the Hirota administration in January 1937 when he engaged in a verbal shouting match against Speaker of the House Kunimatsu Hamada, accusing him of defaming the Army.

With the outbreak of the Second Sino-Japanese War, Terauchi was assigned combat duty and was given command of the North China Area Army in August 1937.

As the war in the Pacific drew to a close, a British Intelligence Liaison Officer, Major Richard Holbrook McGregor, was sent by Admiral Mountbatten to Saigon to verify that Count Terauchi was indeed in a hospital and unable to make the flight to RAF Mingaladon Airfield to personally discuss terms of a cease-fire.

Instead, the remaining 680,000 Japanese soldiers in Southeast Asia were surrendered on his behalf in Singapore on 12 September 1945 by General Seishirō Itagaki.

Yamashita felt otherwise, writing in his diary that "... that damn Terauchi lives in luxury in Saigon, sleeps in a comfortable bed, eats good food and plays shogi".

On 11 June 1946, Terauchi became angered by a report of a Kempeitai lieutenant colonel who had threatened to disclose Japanese war crimes to the Allies, and he suffered a second massive stroke and died early the next morning.