The son of a South African rugby union international, Smith began his career as an amateur with Hitchin Town in the early 1930s, playing in one of that club's finest teams and helping them to the Spartan League title in 1935, before turning professional when he joined Millwall later that year.
In the 1937–38 season Smith enjoyed even greater success, as Millwall hurtled to the Division Three South title, while also claiming the London FA Challenge Cup by defeating Crystal Palace.
Smith became Millwall's last full England internationalist in 1938, when he was selected for two games in seven days in November, despite only playing in the English second tier at the time.
In 1948, Smith was appointed player-manager of Corby Town upon the club's formation but left after only a couple of months for family reasons, returning north to Dundee where he became a coach.
In the summer of 1959, Smith was approached by old club Millwall to replace Jimmy Seed as manager, and he returned to the Londoners for the beginning of the 1959–60 season.