He was born in Camden, South Carolina, to James Cantey and Martha Whitaker.

In 1813 Cantey served in Captain John Irwin's cavalry company in the Creek Indian hostilities.

In 1836 General Cantey recruited a company of mounted infantry for three months duty in the Second Seminole War.

[1] In 1843 he was elected Adjutant and Inspector General of South Carolina and served in that position until 1853.

General Cantey's grandson, Patrick Henry Nelson II, founded The Nelson Law Firm, was a member of the South Carolina House of Representatives, and was the Fifth Circuit Solicitor and the president of the South Carolina Bar Association (1911–1912).