The house was used by the caretaker of Taylor Park for many years and was leased to the Sheboygan County Historical Society in 1949.

The Historical Society removed the jail equipment, including the bars from the windows and opened it as the county museum in 1954.

Some of the exterior architectural details were removed or "updated" at that time and everything received a fresh coat of paint.

Major restoration work took place in 2006 to clean and tuck pointing the brick and included the removal of the 1950s front porch and building a replica of the 1850s formal front porch, repair of wood architectural features as well as the removal of the fire escape.

[3] Because the David Taylor House is on the National Register of Historic Places, there are specific guidelines required in the restoration work.