Born in Fauquier County, Virginia, Kemper and his brothers Nathan and Samuel settled in Feliciana Parish, near Baton Rouge, Spanish West Florida, shortly after 1800.
Expelled from the province by the Spanish authorities in a dispute over land titles, the Kemper brothers organized a small force in the Mississippi Territory and returned, declaring West Florida to be independent.
They attempted to capture Baton Rouge in 1804, but were defeated, having failed to gain the support of local Anglo-American settlers.
Most of the latter were satisfied with Spanish rule on account of Spain's liberal land grants and its protection of slavery.
He was more fortunate than his colleagues, who were seized by the Spanish authorities and sent as prisoners to El Morro, in Havana, Cuba.