Robinson stood 6 feet 1 inch (1.85 m) tall, weighed 195 pounds (88 kg) and batted and threw right-handed during his playing career.

With the exception of a three-year (1943–45) tour of duty in military service during World War II, he caught in the International League with Rochester and the minor league edition of the Baltimore Orioles through the middle of the 1949 season, when he was acquired by the Boston Red Sox' Louisville Colonels farm club.

In 1953, he received his first managerial assignment as skipper of the Bosox' Salisbury Rocots Class D affiliate in the Tar Heel League.

In 1957–58, Robinson managed Double-A clubs for Boston in the Texas League (with the Oklahoma City Indians) and the Southern Association (with the Memphis Chicks).

However, a housecleaning in the Red Sox front office at the close of the 1960 season resulted in the departure of the team's farm system director, Johnny Murphy, and Robinson joined the New York Yankees for two seasons, managing the Amarillo Gold Sox to the 1961 Texas League pennant and helming the Triple-A Richmond Virginians in 1962.