Born in Hanover, Pennsylvania, the left-handed Spencer was recognized for his excellent fielding ability, but also served in later years as a designated hitter.

Spencer began the 1969 season assigned to the Hawaii Islanders, but with former All-Star Dick Stuart not panning out at first base, he was back with the Angels by May.

Spencer was dealt along with Lloyd Allen from the Angels to the Texas Rangers for Mike Epstein, Rich Hand and Rick Stelmaszek on May 20, 1973.

[3] Despite the fact that Spencer committed just one error in 1973 and one in 1974, he began seeing more time at DH with Mike Hargrove assuming most of the first base duties.

[4] The reunion lasted only a day as he was sent with Morris Nettles to the Chicago White Sox for Bill Melton and Steve Dunning on December 11.

During spring training 1981, Spencer was dealt to the Pittsburgh Pirates for Jason Thompson, however the trade was nixed by Commissioner Bowie Kuhn.

[12] Spencer was traded along with Tom Underwood from the Yankees to the Oakland Athletics for Dave Revering, Mike Patterson and minor-league left-handed pitcher Chuck Dougherty on May 20, 1981.

The night before his death, Spencer played first base in a charity baseball game benefiting the Joe DiMaggio Children's Hospital in Hollywood, Florida.