Harlan was born at Evansville, Indiana, graduated from Princeton University in 1883, and studied law in the office of Melville W. Fuller in 1884 to 1888.

From October 1888 to 1889, he served as the first law clerk to Chief Justice Fuller.

[1][2] In 1894, alongside Clarence Darrow and Stephen S. Gregory, Harlan represented Patrick Eugene Prendergast (the assassin of Chicago Mayor Carter Harrison Sr.) in petitioning for a jury to determine his sanity in order to challenge his conviction to the death sentence.

[3] In 1901, President Theodore Roosevelt nominated Harlan as Attorney General of Puerto Rico and he served until 1903.

[4][5] He became a member in 1906, and chairman in 1914, of the United States Interstate Commerce Commission.