In 1840, his family moved to Lake County, Ohio, where he became a pupil at the Western Reserve Teachers' Seminary,[1] a Mormon teaching school in Kirtland.

[2] Leaving school at the age of fifteen, he became a drover for two years, after which he moved to Massachusetts where he worked for his uncle in the fishing industry at Gloucester.

He was on the Starlight at Shreveport, 700 miles (1,100 km) from New Orleans when it was seized by a Confederate committee, who aimed to use it to blockade the river against Union forces.

Having heard of the Union Army's capture of New Orleans on April 25th,[1][3][4] Whitley, the "mulatto" second cook, and another "liberty loving African", stole the steamer's yawl.

[7][8] Whitley, whose successful arrest of 12 Klansmen in Georgia for the murder of a leading local Republican official had led to his appointment by Grant, used talented detectives who infiltrated and broke up KKK units in North Carolina and Alabama.

[8][11][12] The D.C. Supreme Court subsequently found the grand jury which indicted Whitley and others to have been illegally drawn, and the Attorney-General ordered nolle prosequi for the case.