Johnson also filed an anti-discrimination suit against Illinois Central Railroad in the mid-1960s after he was turned down for a special agent position.
Johnson won the suit and became the first African American special agent.
Johnny Washington, a former Negro league player and friend of Johnson, said: "Duty lived on the same block as Charlie and really took a liking to him."
Johnson spent his time in the Negro league barnstorming the United States and Canada.
Johnson married in 1942 and, at his wife's insistence, quit baseball in 1944.