Stationed in South Carolina at the war's end, he accepted the position of superintendent of education for the Freedmen's Bureau.

[5] He also served on the Board of Regents of the Normal School from 1874 to 1877, located on the University of South Carolina campus.

Upon the readmission of South Carolina to the Union, Whittemore was elected as a Republican to the Fortieth and Forty-first Congresses and served from July 18, 1868, to February 24, 1870, when he resigned[3] under the eminent threat of expulsion from the House.

[3] Some of his constituents claimed they had a right to vote in whoever they wanted, as long as they "met the constitutional qualifications of citizenship, age, and residence."

In response, Representative John A. Logan from Illinois stated that the House does have the right to reject a man of his character.

[6] During the latter part of the Reconstruction era (1870 to 1877), there was increasing violence by Red Shirts, paramilitary insurgents who worked to suppress black voting.

[6] Democrats then regained power in the state legislature and began to pass laws to restrict voter registration and reduce the civil rights of freedmen.

[6] In 1876, South Carolina representative Alfred Rush, an African American freedman, was assassinated near his home in Darlington County.

Whittemore wrote a letter to Governor Daniel Henry Chamberlain expressing concern.

He was interred in the Salem Street Cemetery in Woburn,[3] as was his wife Mandanna (also Mandana) Dora Stone.