The church was built near the north end of the lake Kviteseidvatnet because centuries ago, the waterways were the main transportation routes throughout the municipality, so this was a prime location.

[8][10] During the 19th century, a small tower was built on the roof of the nave and the two windows on the south wall were greatly enlarged.

In the late 1800s, it was decided to move the Kviteseid Church to the village of Kviteseid and to compensate for moving it further to the northwest, the parish would also build a brand new church at Fjågesund to serve the eastern part of the parish.

Because of the small size of the parish, it took many years to raise the money needed to construct both new churches.

After this, the Old Kviteseid Church was closed down and it was left untouched for several decades before being carefully restored in 1929 and again in 1969.

During the restorations, the 19th century white paint was removed to uncover the rosemåling that had been covered up.

Several former parish priests of Kviteseid (and provosts of Øvre Telemark) are buried under the church floor, among them Peder Povelsson Paus.