Born in New London, Connecticut, Gardner was signed by the Giants in 1945 and came up with them on April 22, 1954, but he could not break into the contending team's lineup.

Gardner wound up as a utility infielder with 1961 Yankees, winning the 1961 World Series with them against the Cincinnati Reds.

In all or parts of ten seasons, Gardner batted .237 with 41 home runs and 271 RBIs in 1,034 games played.

He was promoted to manager on May 23, 1981, replacing Johnny Goryl, and served until June 21, 1985, never leading Minnesota to the playoffs and avoiding a losing record only once (1984, at 81–81).

Gardner initially signed as the Royals' 1987 third-base coach, but terminally ill Royals manager Dick Howser, diagnosed with a malignant brain tumor during the summer of 1986, was forced to retire during spring training, and Gardner was promoted to fill the vacancy.

Gardner's son, Billy Jr., a former minor league infielder, is a manager in the Miami Marlins' farm system, currently with the Beloit Sky Carp.