Born in St. Louis, Missouri, Sweetser later attended Phillips Exeter Academy and Yale University.

He received golf lessons from noted Siwanoy Country Club professional Tom Kerrigan.

Amateur at the age of 20, defeating Bobby Jones, 8 and 7, in the semi-final and then Chick Evans, 3 and 2, in the final match.

In 1926, Sweetser won the British Amateur at Muirfield, defeating Fred Simpson, 6 and 5, in the final match.

The 1904 winner, Walter Travis, was a naturalized American citizen born in Australia, but Sweetser's victory was the first time an American-born golfer had won the tournament.