Often referred to as the "Prince of Fullbacks" due to his sportsmanship and ability, Spencer is considered one of the greatest Aston Villa captains of all time.

[1] Spencer played local amateur football for a number of clubs in his local area before joining Football League First Division side Aston Villa from Birchfield Trinity in April 1892 as an amateur, turning professional two years later in June 1894.

A one-club man, he spent 13 years with Villa, taking the 1901–02 season off in order to rest his knee and ankle for twelve months in the hope of recovering from an injury.

Over the course of his career, Spencer gained a reputation as a fair player and was well known for his good sportsmanship which earned him the nickname the "Prince of Fullbacks".

A sports reporter who watched Spencer play said this about him: "He was the outstanding example of what Aston Villa tradition on the football field came to mean.