Joseph was born free in about 1846 in Florida and traced his ancestry to Spain, France, Africa, and Cuba.
[2] During reconstruction (1865-1877), Joseph was the president of the Mobile branch of the Union League and involved in civil rights.
He eventually founded and edited four newspapers in Mobile and Montgomery from 1870 until 1884[4] In 1872 he decided to oppose the reelection of Benjamin S. Turner to the United States House of Representatives.
During the campaign, Turner accused Joseph of having been a "secret agent of the rebel government" during the Civil War.
[11] In March 1880, Joseph spoke before Congress at the Exodus Investigation, testifying that blacks in Alabama were kept from the polls and were violently prevented from leaving the state.
He was aggressively cross-examined by Zebulon Baird Vance, who questioned why blacks should be allowed to vote if they were intimidated away from the polls in spite of outnumbering whites in some areas by a count of eight to one.
Garfield survived in intensive care for 11 weeks, and on account of the political turmoil, Joseph's commission was never confirmed.
He was pardoned by Governor William D. Jelks, reportedly because when committing the crime Joseph was under the influence of morphine and did not know what he was doing.