The municipality, at a height of about 1,100 m (3,600 ft), is the main settlement in the East Tyrolean Hochpustertal, the eastern part of the Puster Valley, stretching from Lienz up to the border with South Tyrol, Italy, in the west.

The weather in Sillian is extremely changeable due to its location, and it can quickly change from warm and balmy to sub-zero.

Upon the extinction of the line in 1500, Sillian was seized by the House of Habsburg king Maximilian I and incorporated into the County of Tyrol.

By the 1919 Treaty of Saint Germain, it became a border town close to the South Tyrolean lands of the Italian kingdom.

Young Richard Strauss from 1872 onwards spent several summers at Sillian; he is purported to have played the church organ as well as the hotel piano of the Schwarzer Adler (Black Eagle) inn, and even to have composed here.