A native of Davenport, Iowa, Baker starred on the basketball and track teams at Davenport High School, and played sandlot baseball, then went into the United States Navy, being stationed at Ottumwa Naval Air Station and Iowa Pre-Flight School.

After his discharge from the Navy, he played for the Kansas City Monarchs of the Negro American League as their regular shortstop during 1948 and 1949.

Bobby Bragan, manager of the Angels’ chief rivals, the Hollywood Stars, said Baker was "as good a shortstop as I’ve ever seen – and that includes Pee Wee Reese.

"[1] The Cubs purchased Gene Baker's contract and he made his major league debut September 20, 1953.

[3] In 1962, he became the first black coach in Organized Baseball when the Pirates named him player-coach of their Triple-A International League affiliate Columbus Jets.