In 1791, he removed to Jerusalem, then in Ontario County, New York, settling in that part which was separated in 1803 as the Town of Vernon, later renamed the Town of Benton, and engaged in agricultural pursuits.

He was a member from Ontario County of the New York State Assembly in 1819.

Spencer was elected as a Democratic-Republican to the 17th United States Congress, holding office from December 3, 1821, to March 3, 1823.

He was the delegate from Yates County to the New York State Constitutional Convention of 1846.

He was buried at the Lake View Cemetery in Penn Yan, New York.