It then moved to its permanent headquarters in François Baillairgé's 1808 city gaol at 44, Chaussée des Écossais (then Rue Saint-Stanislas) after architect Joseph-Ferdinand Peachy remodelled the building.

In addition to persistent financial problems, the Anglo-Protestant population of Quebec City was too small to justify an institution of its kind.

Past financial contributions have ensured the survival of the Literary and Historical Society of Quebec at a precarious time in the early 20th century.

The school was also a local pioneer in terms of women's education, allowing female students to study for a BA in co-ed classes from 1885.

A few people who later went on to illustrious careers taught at the college, namely Edwin Hatch, James Douglas Jr., and George Mercer Dawson.