Over the years, the village of Kviteseid grew up around the northern end of the lake Sundkilen and the old church stood about 6 kilometres (3.7 mi) to the south of there, along the lake Kviteseidvatnet.

In the late 1800s, it was decided to move the Kviteseid Church to the village of Kviteseid and to compensate for moving it further to the northwest, the parish would also build a brand new church at Fjågesund to serve the eastern part of the parish.

Because of the small size of the parish, it took many years to raise the money needed to construct both new churches.

New designs were then obtained from the architect Haldor Larsen Børve.

After this, the Old Kviteseid Church was closed down and it was left untouched for several decades before being carefully restored.