Born in Chatham, New York, Penn and his family moved, in 1841, to Walworth County, in the Wisconsin Territory, and settled on a farm near the present site of the town of Lyons.

[3] At the outbreak of the American Civil War, Lyon formed a company of volunteers in Racine County and was elected their Captain.

[2][3][4] The 8th Wisconsin was ordered to Missouri, in the Western Theater of the war, and attached to General Ulysses S. Grant's Army of the Tennessee.

[5] In August 1862, by order of Governor Edward Salomon, Lyon was made Colonel of the 13th Wisconsin Volunteer Infantry Regiment.

[4] The 13th Wisconsin saw very little combat, and was primarily tasked with securing trains and other logistics supplying frontline forces in the western theater.

In the 1874 case of Attorney General v. Chicago & Northwestern Railroad Company, Justice Lyon wrote with the majority that sustained the legislature's power over corporations operating within the state.

It had been a common practice in Edgerton, Wisconsin, for teachers in the public school to read passages from the King James Bible.

In 1886, Roman Catholic parents complained to the school board about this practice, which they saw as teaching a sectarian and inaccurate version of the bible.

Weiss and others vs. District Board, etc., later referred to as the "Edgerton Bible Case", Justice Lyon wrote the unanimous opinion of the court that overturned the 12th Circuit opinion and ruled that the Edgerton public school practice was sectarian instruction and, therefore, violated Article X, Section 3 of the Wisconsin Constitution, which explicitly prohibits sectarian instruction in state public schools.

He required two surgeries to drain the pus and remove degenerated bone tissue, leaving him with life-long weakness in his leg.

The case was re-tried at the Circuit Court, and again found in favor of the plaintiff, this time awarded damages of $2,500 (approximately $71,000 adjusted for inflation to 2019).

He died at his daughter's home in the Edenvale neighborhood of San Jose, California on April 4, 1913, and was buried at Oak Hill Memorial Park.