Wharton's early life is unclear, during his boxing career he described himself as being born in Tangier, Morocco on 3 March 1813 and that his mother was a peasant, while his father was a sailor.

[1] Wharton said he started working as a cabin boy on the a ship called Hopewell at the age of twelve.

[4] Wharton won his first three career fights against Tom McKeevor in 1833, "the Herefordshire Pippin" Evans in 1834, and "the Hammersmith Cowboy" Jack Wilsden in 1835.

[8] Throughout 1837, Wharton won matches against Sandy MacNeish in Scotland and William Renwick in England.

[2] On 25 April 1856, Wharton died in Liverpool, England from complications related to tuberculosis.