When a 1918 fire destroyed the Tavern, the Hospital was rebuilt across the street and continued to serve the community until the 1970s.

[10] To preserve the building, Morristown historian Julia Keese Colles, a founding member of the Women's Board of the New Jersey Historical Society, arranged to move the building from the Morristown Green to her estate on Mt.

Though extended and enlarged, it is still the same building and retains many of the distinctive features which characterized it when the residence of Washington.

[11]Colles renovated the Tavern into the "Colonial House," which was "a residence for summer boarders who came to Morristown as a vacation area.

"[12] After a few years, it was sold at a public auction in 1890, where it was purchased by the Catholic Church to become the All Souls' Hospital.

According to Caroline Foster and Barbara Hoskins, "The ballroom was transformed into the chapel while the dining room was the hospital ward.

"[12] On January 4, 1892, a constitution and set of by-laws were discussed in a general meeting in Bayley Hall, Morristown.