He was born in South Bend, Indiana, spent his early years in Evanston, Illinois, and moved to Los Angeles with his family as a boy.

After attending Los Angeles High School, where he was an All-City third baseman in 1943, and military service during World War II, Walsh joined the Brooklyn Dodgers organization in 1948 as a member of the front office staff of the Fort Worth Cats, Brooklyn's farm club in the Double-A Texas League.

When the Dodgers acquired the Los Angeles franchise of the Pacific Coast League in February 1957 — the precursor of the Brooklyn club's historic shift to the West Coast, which would follow at the close of the 1957 season — Walsh, as an Angeleno, became president of the minor league team and a liaison between the Dodgers and the city of Los Angeles.

Walsh fired Bill Rigney, the only manager in the club's Major League history to that point, and replaced him with Lefty Phillips, who had just joined the Angel front office after a long career as a scout and pitching coach with the Dodgers.

Then during the 1969–70 offseason, Walsh acquired the highly talented but controversial outfielder Alex Johnson in an inter-league trade with the Cincinnati Reds.

During the 1970–71 offseason, Walsh traded for another heavy-hitting outfielder, Tony Conigliaro of the Boston Red Sox, who had hit 36 home runs, fourth in the American League, during 1970.

Johnson was handed a season-ending ban on June 26 (which he contested in court); his year ended after only 65 games and saw his average decline almost 70 points, with two home runs.