James Grafton Spencer (September 13, 1844 – February 22, 1926) was a U.S. Representative from Mississippi.
Born near Port Gibson, Mississippi, Spencer attended private schools and Oakland College in 1861.
During the Civil War enlisted in the Confederate States Army as a private in Cowan's battery of Light Artillery.
He returned to his home and engaged in agricultural pursuits.
He died in Port Gibson, Mississippi, February 22, 1926.