His father died when he was three years old and his mother, a nurse, moved with him to Brooklyn, where he attended Erasmus Hall High School.
He often told a story of covering a game at Manhattan College where the crowd so overwhelmed the tiny gym that he had to climb in through a window, tearing the pants of the best suit he owned.
As this was during the Great Depression and the venue was often going unused, Irish was able to book events without putting up his own money so long as attendance at least covered the nightly rent.
[5] As the Madison Square Garden games became popular showcases and most major college teams were eager to be booked, Irish was able to insist on terms favorable to him.
[5] His promotions of college matchups were important to growing the popularity of basketball at a time before there was an established professional league.
[4] In the fallout of the 1951 CCNY point-shaving scandal, which was centered at Madison Square Garden, New York City District Attorney Frank Hogan issued a report saying "the blatant commercialism which had permeated college basketball" was an underlying cause.