The son of a Union Army veteran of the American Civil War, Penn was born in Mattoon, Illinois, and raised in Batavia, Ohio.

During the Spanish–American War, Penn served as a quartermaster officer at Camp George H. Thomas, Georgia and the Tampa mobilization point before contracting typhoid.

He served on the staff of the Adjutant General of the Army following the war, then commanded the US Disciplinary Barracks at Governors Island, New York until retiring in 1924.

[4] He then took a competitive examination for appointment to the United States Military Academy (West Point) by his district's Congressman, Henry Lee Morey.

[4] The second-place finisher failed his midterm examination in January 1882 and Penn, who would be old enough for admission after he turned seventeen in February, successfully petitioned for the appointment.

[5] They hired a cook to grow vegetables and prepare meals for all three, and went on extensive hunting and fishing trips, which led to them being dubbed “The Three Green Ps”.

[6] During the early years of his career, Penn served throughout the Western United States, including postings to Oklahoma, Wyoming and Idaho.

[6] Selected to command the regiment's 2nd Battalion, he was posted to Fort Logan and spent the summer recruiting soldiers from Colorado, New Mexico, and Arizona.

[6] Penn's regiment arrived in the Philippines in October 1899 and took part in Arthur MacArthur Jr.’s expedition near Dagupan that attempted to capture Emilio Aguinaldo.

[6] In November and December 1899, he took part in Samuel B. M. Young’s operations in Northern Luzon, which resulted in the freeing of over 2,000 Spanish prisoners, as well as several U.S. Navy sailors and American civilians.

[7] When the regiment was dispatched to Texas during the Pancho Villa Expedition, Penn commanded it as it patrolled the Texas-Mexico border from bases at Fort Sam Houston and Camp Eagle Pass.

[7] When the Army began to expand in anticipation of U.S. entry into World War I, Penn was assigned command the newly-organized 37th Infantry Regiment, which he equipped and trained at Fort Sam Houston.

In December 1918, a month after the Armistice with Germany which ended hostilities, Penn returned to the United States and served as commander of Camp Zachary Taylor, Kentucky.

[8] In 1922, Penn was assigned as adjutant of the Third Corps Area at Fort Howard, Maryland,[8] and in the summer of that year performed additional duty as honorary aide-de-camp to President Warren G. Harding in Ohio during celebrations to commemorate the 100th anniversary of Ulysses S. Grant’s birth.