In three years of college football at the University of Oklahoma beginning in 1937, Ivy played both offense and defense for the Sooners, earning All-American honors in 1939 as an end.

His time on the gridiron was interrupted for more than two years by his service in World War II, but he closed out his career in 1947 by helping the franchise to its only NFL title.

He also became known as an innovator in Canadian football, taking advantage of the more wide-open game by using strategies such as the twin fullback system, the quick snap and the short kickoff.

While he publicly stated that he had "not accomplished the desired results", Ivy's decision reportedly stemmed from a growing feud with team Director of Operations Walter Wolfner.

During Ivy's first year, the team again won the conference title with an 11–3 record, but dropped a classic 20–17 double overtime decision in the 1962 AFL Championship to the Dallas Texans.

That success resulted in the CFL Alouettes team seeking his services, but on February 14, 1963, Ivy signed a two-year deal with the Oilers that gave him complete personnel authority.

After two seasons, the last a disastrous 1–12–1 campaign, Owner Wellington Mara enacted a new Club rule stating that all coaching staff must live in the New York/New Jersey area year round.