It is located on the northeastern part of the large island of Hinnøya, along the Andfjorden, about 24 kilometres (15 mi) northwest of the town of Harstad and about 16 kilometres (9.9 mi) north of the village of Kasfjord.

Elgsnes is located at the tip of a narrow peninsula, looking across the Toppsundet strait towards the village of Grøtavær on the island of Grytøya.

A woman's grave from the Merovingian period (600 – 800 A.D.) was richly equipped, including jewelry from the Baltic region and Karelia.

A blacksmith's grave from the Viking Age (800 - 1.050 A.D.) was, when it was opened in the 1950s, considered to be the fourth richest of its kind in Scandinavia.

In 1798, the trade station got a royal license as a guesthouse, so the use of the building for overnight guests started more than 200 years ago.