When his parents returned to the United States, settled in Bath-on-the-Hudson, New York, and opened a paint shop, Palmer joined them and became a house painter.

At the expiration of his three-year enlistment, Palmer re-enlisted, and the regiment was transferred to the V Corps of the Army of the Potomac under General G. K. Warren.

He was seriously injured at the Battle of Five Forks when a shot horse fell on top of him, and his back was cut by the sword of its rider.

He is the maternal grandfather of John Palmer Harcourt Jr, a retired educator who served as assistant secretary to the Governor Nelson Rockefeller.

"Following his death, ownership was transferred to Palmer’s wife, Margaret, but the property was then sold in 1910 to the Vice President of Ryan & Graves Insurance, Samuel C Harcourt.