He gained fame for his bravery on the battlefield, becoming the youngest noncommissioned officer in the history of the United States Army at the age of 12.

He retired from the Army in 1915, having attained the rank of brigadier general in the Quartermaster Corps; he was at that time the last veteran of the American Civil War still on duty in the United States Armed Forces, although others similarly aged and experienced soldiers such as Peter Conover Hains and Albert A. Michelson rejoined the military after American entry into World War I in 1917.

The legend suggests that he came very near to losing his life when a fragment from a shrapnel shell crashed through his drum, knocking him unconscious and that subsequently his comrades who found and rescued him from the battlefield nicknamed Clem "Johnny Shiloh.

Regardless of the time of his entry into service, Clem served as a drummer boy for the 22nd Michigan at the Battle of Chickamauga in September 18–20, 1863, when he was twelve.

After the battle, the "Drummer Boy of Chickamauga" was promoted to sergeant, the youngest soldier ever to be a noncommissioned officer in the United States Army.

[4] Secretary of the Treasury, later Chief Justice of the United States, and fellow Ohioan, Salmon P. Chase, decorated him for his heroics at Chickamauga.

[5] He was included in a prisoner exchange a short time later, and the Confederate newspapers used his age and celebrity status for propaganda purposes, to show to "what sore straits the Yankees are driven when they have to send their babies out to fight us."

After he failed the entrance exam to enter the United States Military Academy, President Ulysses S. Grant appointed him second lieutenant in the 24th Infantry Regiment in December 1871.

Through his military career Clem held the following ranks:[8] In 1963, Walt Disney produced a made-for-TV film entitled Johnny Shiloh, with Kevin Corcoran in the title role.

The film, which mixes historical and fictional narratives, includes numerous U.S. Civil War reenactments, and focuses on what life was like for Union soldiers.