Kemper entered Columbia College at the age of fifteen, where he studied theology under Dr. Henry Hobart and graduated in 1809 as the valedictorian of his class.

Particularly interested in evangelism, Kemper even persuaded his elderly mentor to make a missionary journey to western Pennsylvania during which also he founded St. Matthew's Episcopal Church in Wheeling, West Virginia.

[1] In 1835, the Episcopal Church's General Convention decided to consecrate missionary bishops to preach the Gospel west of the settled areas.

Kemper constantly urged outreach to the Native American peoples, and translations of the Scriptures and the services of the Church into their languages.

There Kemper ordained William Adams and James Lloyd Breck, two young recruits from the East who helped him establish Nashotah House Seminary, on October 9, 1842.

[7] Kemper Hall, an Episcopal school for girls in Kenosha, Wisconsin that was named after him, is also listed on the National Register.