Rutherfoord graduated from Princeton University in 1816, studied law, and was admitted to the bar.

Originally a Democratic-Republican, and later a Whig, and then a Democrat, he served in the Virginia House of Delegates from December 1826 to March 1834.

In 1841 Governor Thomas Walker Gilmer resigned to accept election to a seat in the United States House of Representatives.

His place was taken by John M. Patton, who was first in the line of succession as the Executive Council's senior member.

Upon his resignation, he was succeeded by another Executive Council member, John Munford Gregory, who completed the term to which Gilmer had been elected.