Even as a schoolboy he was introduced to locomotive design by his father, the chief mechanical engineer (Maschinendirektor) of the Imperial and Royal Southern Railway (k.u.k.

From 1893 to 1916, Karl Gölsdorf was the chief design engineer of the Imperial Royal Austrian State Railways (kaiserlich-königlichen österreichischen Staatsbahnen or kkStB) and developed 25 different classes of steam locomotive (in 47 variants) during his career.

Notable 'special' designs included the rack railway locomotives of Class 269 for the Erzbergbahn and the narrow gauge Yv for the Ybbstalbahn.

Karl Gölsdorf died unexpectedly on 18 March 1916 at Wolfsbergkogel whilst staying in Semmering, Austria, of an acute throat infection and not, as incorrectly reported in media at the time, from the consequences of an accident with a locomotive.

The saying that "you can't save a tonne of weight on a locomotive - but you can save a kilo in a thousand places" (Man kann an einer Lokomotive nicht eine Tonne Gewicht einsparen - wohl aber an tausend Stellen ein Kilo) is attributed to Gölsdorf.