Richard A. Wilkins, a Virginian who had returned from operating a sugar plantation in Louisiana to educate his children, bought the property in 1860 for $20,000.

His wife operated a hospital for recovering soldiers in the mansion during the Civil War, and their son watched battles from the cupola.

[5] After the war ended, Catholic Bishop John McGill requested nuns from the Archdiocese of Baltimore, to pray for the devastated city as well as to educate girls.

[6] Although the Sisters of the Visitation were generally a contemplative order in Europe, for more than sixty years they ran an elite boarding and day school on this site, which they named Monte Maria.

In 1895, following a donation from Thomas Fortune Ryan, the Visitation order built a Romanesque revival style brick chapel and service building to the east of the original Taylor House.

A bequest from the Mother Superior in 1927 allowed the Visitation nuns to close the school and concentrate on their contemplative activities, so the sisters renovated the dormitories into individual cells.

In 1952 a print shop was added in a cinder block wing that replaced a small porch on the building's west side.