The building's name arose as the place was home to several authors, including Devorah Baron, Yosef Aharonovich, Asher Beresh, Yehuda Burla, and more.

During World War I the building stopped serving the purpose of housing authors, and became an ordinary residence until it was abandoned in the 1960s.

The museum was opened in May 1998, and the artwork was donated by Nahum Gutman's family, who had wanted to perpetuate his works.

In September 2009 a new art exhibition division was opened, named after Mori and Ireen Pargament, as well as an events hall for up to 70 people and a new gift store.

In 2013, board member and lawyer Yitzhak Shviger resigned, claiming that the foundation had become "a stage for Palestinian propaganda".