He grew up in Pennsylvania, and played football in college at Waynesburg College in Waynesburg, Pennsylvania and went on to join the Cleveland Rams in the National Football League (NFL) as a center following a stint in the United States Army during World War II.

[7] Scarry suffered a knee injury near the beginning of the 1945 season, but soon returned to action as the Rams, led by quarterback Bob Waterfield, won the NFL championship.

[2] The Rams moved to Los Angeles after the 1945 season, and Scarry, along with teammates Chet Adams, Tommy Colella, Don Greenwood and Gaylon Smith, decided to stay in Cleveland and play for the Cleveland Browns, a team under formation in the new All-America Football Conference.

[11] While Scarry was playing for the Browns, he was named head basketball coach at Western Reserve, where he had taken classes between games and in the offseason.

[2] Toward the middle of the season, Cleveland coach Paul Brown began to use him as the defensive leader, letting him call the unit's formations.

[2] In December, Scarry's Western Reserve basketball team played its first games; he had missed numerous practices because of his duties with the Browns.

In September 1947 he received a bachelor of science degree from Western Reserve, completing an educational career at Waynesburg that was cut short by the war.

[18] Scarry borrowed Paul Brown's coaching techniques at Western Reserve, instituting well-organized practices there.

"[19] With no good passer or runner and a lack of depth, Western Reserve's Red Cats performed poorly in Scarry's first season, but he was praised for making the most out of a thin squad.

[27][28] After seven seasons at Cincinnati, Scarry got his third head coaching job, for the Yellow Jackets at Waynesburg, his alma mater.

[32] Scarry continued to act as the line coach under Graham for the college all-stars in the offseason during his tenure at Waynesburg.

Scarry then scouted briefly for the San Francisco 49ers, Los Angeles Rams, and Dallas Cowboys before taking a job in 1970 as the defensive line coach for the Miami Dolphins under Don Shula, a former Browns player.

[23] Miami reached the Super Bowl five times while Scarry was a coach there, winning consecutive championships in the 1972 and 1973 seasons.