[1] After graduating from Union College in 1806, Spencer practiced law and held various positions, including master of chancery, postmaster, and attorney general.

Spencer faced challenges in his role as Treasury Secretary, including a deficit, tariffs, and the development of a plan for a Board of Exchequer.

[8] During the War of 1812, Spencer served in the United States Army where he was appointed brigade judge advocate general for the northern frontier.

[1] In 1826, Spencer served as a special prosecutor to investigate the disappearance of William Morgan who was arrested, kidnapped and murdered for exposing secrets kept by Freemasons, thus sparking the Anti-Masonic movement.

[11] He also recommended that the government adhere to arrangements made by Army commanders in the field for compensation of the Creek Indians, who had been forced to move west of the Mississippi.

In 1842, his nineteen-year-old son, Philip Spencer, a midshipman, was executed without court-martial along with two other sailors aboard the brig USS Somers for allegedly attempting mutiny.

As Treasury Secretary, he was preoccupied with the tariff and believed that the deficit and other federal expenditures should be funded by duties on imports rather than by internal taxation, something he was forced to announce for the fiscal year in 1843.

He also continued to develop a plan, originally initiated by Forward, for a Board of Exchequer to keep and disburse public funds raised by duties.

The Exchequer bill, which reflected continuing interest in some form of independent treasury system, failed due to a political conflict in the United States Congress.

[13] As one of few northerners in an administration dominated by southern interests, Spencer had found it increasingly difficult to serve in his cabinet post and resigned as Treasury Secretary in May 1844.

Together, they were the parents of several children, many of whom died in infancy or under unfortunate circumstances:[14] In Canandaigua, he lived for 36 years in a house at 210 Main Street, that was built by General Peter Buell Porter (1773–1844), the United States Secretary of War under John Quincy Adams, in about 1800.