King Christian IV ordered the establishment of the inn on 13 January 1624 so that visitors to his new Knight academy, which had opened the previous year, would not have to stay within the school grounds.

Architect Frederik Carl Christian Hansen (1858–1823) designed an expansion of the building in 1899–1900 with two lateral wings, in one storey and with large gabled dormers, which project from its rear side.

[4] The Joachim Burser Physic Garden is located to the rear of the museum .

It is a 17th-century style physic garden recreated by the local Society of Horological History in collaboration with the museum.

It is named after German-Danish botanist and physician Joachim Burser (1583-1639) who was called to Denmark by Christian IV in 1625 where he was appointed as professor at Sorø Academy and served as the second apothecary in the town of Sorø.