His first immigrant ancestor to America was the youngest son of a British member of Parliament named James Palmer (1585–1658), who moved to Virginia in the 1600s.

After his parents' divorce,[1] his father's move to Illinois, and his mother's death, Joseph was raised by his grandparents.

[2] At the beginning of the American Civil War in 1861, Palmer chose to follow his home state and the Confederate cause.

[4] Temporarily unfit for active field service, Palmer was appointed to district command in the Department of Tennessee beginning on November 18, 1863.

[6] On March 19, 1865, Palmer fought during the Battle of Bentonville and was again wounded, but remained in brigade command until April 26.

Palmer died in 1890 at his home in Murfreesboro, Tennessee, and his body was buried in the city's Evergreen Cemetery.