His father, Littleton, was a policeman, but abandoned the family, leaving Ashford's mother Adele to raise Emmett and his brother Wilbur.

[1] Ashford served in the Navy during World War II, and was inspired to become the first black major league umpire while stationed in Corpus Christi, Texas, when an announcement came on the radio that Jackie Robinson had broken baseball's color barrier.

[3][4] In 1951, Ashford took a leave of absence from his Santa Ana, California post office job, where he moonlighted as a Santa Ana municipal league softball and National Night Ball League of Southern California umpire.

[5] Ashford spent 12 years in the PCL, and became known for his exuberance, showmanship and energy, even interacting with the crowd between innings.

In 1963, Ashford was named the PCL's umpire-in-chief, making him responsible for training crews and advising the league on disputed games or rules.

[1] By the early 1960s, many West Coast sportswriters began to suggest that Ashford be promoted to the major leagues.

The Sporting News stated that "For the first time in the history of the grand old American game, baseball fans may buy a ticket to watch an umpire perform.

[7] Ashford was one of the only umpires fiery Baltimore Orioles manager Earl Weaver was ever nice to during a game.

Upon his death, Bowie Kuhn issued a statement, saying, "As the first black umpire in the major leagues, his magnanimous nature was sternly tested, but he was unshaken and uncomplaining, remaining the colorful, lively personality he was all his life.