At first a Whig, he joined the nascent Republican Party and was elected to the Ohio House of Representatives from Madison County in 1857 by a majority of 24 votes.

His senatorial term coincided with the 1860 United States presidential election and the ensuing secession crisis that preceded the American Civil War.

In line with the conservative position staked by President James Buchanan, he denounced secession as "revolutionary," disclaimed any intent to interfere with slavery in the states, and called for the repeal of "enactments ... conflicting with, or rendering less efficient, the Constitution or laws of the United States."

He narrowly defeated Aaron Harlan, the candidate of the Radical Republicans, and served the remainder of Corwin's term.

[5] In October 1864, he denied rumors that he would vote for the Democratic nominee for president, General George B. McClellan, and declared his support for Lincoln's re-election.

In 1893, Harrison was again considered for a judicial appointment, this time to the United States Supreme Court; on this occasion, he was passed over due to his advanced age.

[10] He praised President Theodore Roosevelt and Attorney General Philander C. Knox in connection with the Northern Securities case, and predicted the court would find in favor of the administration.