Hoge won a seat as a Republican to represent the 3rd congressional district after he successfully challenged the election of Democrat J.P. Reed to the Forty-first Congress.

Since the Republicans controlled Congress, the two Democrats elected from South Carolina were unseated and Hoge filled the seat on April 8, 1869 and served the remainder of the term until March 3, 1871.

In 1874, Hoge waged another run for Congress to represent the 3rd district and he defeated Samuel McGowan, a Conservative Party candidate, to win the seat.

In 1870, Hoge appointed James Webster Smith, a former slave, to the United States Military Academy marking the first time an African-American had been admitted.

[2] Upon the completion of Hoge's term in 1877, South Carolina Republicans were in a state of disarray following Wade Hampton's victory in the 1876 gubernatorial election.