He made his Torquay debut in a 3–1 win away to Walsall on 5 December 1953, with the regular left-back Jimmy Drinkwater out of the side.
He started the 1956–57 season out of the first team with Anderton and then regular centre-half John James playing at left-back.
However, Smith soon regained his place and remained a regular for the remainder of the season as Torquay lost out on promotion to Division Two on goal average from Ipswich Town.
He played 188 league games for Torquay, scoring just once, before leaving in July 1961 to join Bristol City.
He played just once for the Ashton Gate side before leaving league football to settle down in Torquay, where he became an avid pigeon racer.