In 1873 he and his brother Thomas won a double sculls race of four miles in a time of thirty-three minutes.
In 1883 he won a professional match race at £10 a side by winning a light skiffs event at Sackville.
The following year at the same location he rowed George Solomon for £50 a side in light skiffs over three miles and won.
His next major race was in October 1884 over three and a quarter miles and for £200 a side against Neil Matterson, a man Kemp, and Bill Beach, were later to row for the World Title.
Over the next two years he won a number of smaller races and handicap events and collected some useful prize money.
Kemp had no great success in England and returned to Australia where he continued to have wins in some of the local regattas and small match races.
He wanted to prove he was worthy and so accepted a title challenge from Thomas Clifford, a fellow Australian.
The race was set down for 11 February 1888 on the Parramatta River, Sydney, over the usual distance of three and a quarter miles and for a stake of £200 a side.
No effort by Clifford made any difference and Kemp crossed the line about six lengths ahead.
This time the stake was £500 a side which was a huge sum of money in those days compared to ordinary wages.
The men got away to a clean start with Kemp taking a small lead early although Hanlan soon levelled.
He had been challenged for the Title by another Australian, Henry Ernest Searle and the race was agreed to be run on 27 October 1888, on the Parramatta.
At the start Searle took the lead and although Kemp made great efforts he could never overtake the leader who won by about twenty lengths in a time of 22m.44s.
Searle did not have any Title defences in Australia but went to England where he defeated a Canadian challenger, William Joseph O'Connor, on the Thames River in 1889.
As Searle was returning on a ship to Australia he contacted typhoid fever and died in Melbourne in December 1889.
Neil Matterson had been Searle's coach and was now of the opinion that he could beat Kemp and thus challenged for the World Title.
Despite a brilliant spurt by McLean he was unable to pass the leader who "won as he liked" by six lengths in a time of 23m.46s.
He sculled well and despite the best work of Kemp he was unable to overtake McLean who finished six lengths ahead in a time of 22m.13s.
Kemp was dissatisfied with the result and offered to row either McLean or Jim Stanbury for £500 a side after April the following year.
Nearby streets are named after some of the other Australian World Sculling Champions, vis, Beach, Searle, Stanbury, and (George) Towns.