Robert Smith, whose name has been given to the house, was the first Episcopal bishop of South Carolina and was also himself the first president of the College of Charleston.
In 1770, streets were laid out through the lands, and a parsonage for St. Philip's Episcopal Church was planned for four acres.
The house's property included the land all the way to Wentworth St.
It is a large, two-story, double house of Carolina brick on a high foundation.
[4] When College of Charleston president George Benson moved out of the house in August 2014, maintenance workers discovered water infiltration problems which would require approximately $100,000 in repairs; although Glenn McConnell was sworn in as the new president of the college, he has not begun occupying the house while awaiting the repairs.[5][when?]