Located on the Vestfold Line, the tunnel was built as part of the 7.8-kilometer (4.8 mi) double-track high-speed segment from Barkåker to Tønsberg.

The Jarlsberg Tunnel runs roughly north–south through Frodeåsen, a hill just north of the town center of Tønsberg.

[3] The Jarlsberg Tunnel constitutes the southernmost part of the 7.8-kilometer-long (4.8 mi) double-track segment of the Vestfold Line between Barkåker and Tønsberg.

[1] During this period, there was very little investment funding for railways; thus the Vestfold Line was split into a series of small segments, each which was planned individually.

[8] In September 1999, the government presented their proposal for National Transport Plan 2002–11, which included three segments on the Vestfold Line: Holm–Nykirke, Barkåker–Tønsberg and Farriseidet–Porsgrunn.

[11] In response, Minister of Transport Torild Skogsholm stated that she was considering financing the project as a public–private partnership paid through a surcharge on tickets fares.

NSB's Tom Ingulstad called the plans "troublesome" and stated that the trains would have nearly no time or reliability-gains from the investment.

If the authority instead had built more double track in connection with the existing segment at Sande, trains could more efficiently catch up any delays before reaching Drammen.

The National Rail Administration had at the time of construction not made any plans for a route south of Tønsberg; estimates from Norsk Bane show that Tønsberg will not be able to allow high through speeds and that a new through line would have to be built with a different right-of-way, entirely avoiding the Jarlsberg Tunnel.

[8] For Jernbane advised against building the Barkåker–Tønsberg segment and instead recommended that the authorities wait until the entire Vestfold Line was planned.

[6] Six bids were issued to building the main segment, which included the tunnel and 2.6 kilometers (1.6 mi) from Tomsbakken to Barkåker Industrial Park.

The bidders were a joint venture between Reinertsen and Leonard Nilsen, Veidekke, Skanska, Hæhre Entreprenør, NCC and Mika.

[22] Construction ran from a crosscut in the center of the tunnel and outwards; average speed was 35 to 40 meters (115 to 131 ft) per week.

This was the most hectic part of construction, as it saw the tracks south of the tunnel be rearranged to allow trains to operate the opposite direction through the loop in Tønsberg.