The building was constructed between the years 1744-45 and was commissioned by the wealthy merchant Christian Ancher.

Situated in what is present-day Fred.Olsens gate 6, close to Oslo Central Station, the building was near a public garden, Paléhaven, that stretched down to the shores of Bjørvika.

King Charles XIV John expanded townhouse greatly, with the old stables being converted into residential areas, all decorated in Empire style.

Paléet was subsequently given the role as the royal residence in Oslo during the era of the Union between Sweden and Norway, a role the townhouse maintained until the completion of the Royal Palace in 1848.

The townhouse was demolished after a damaging fire in 1942 and no trace of the building can be seen today.