Charles François Xavier Baby (19 June 1794 – 6 August 1864), was the eldest son of François Baby and received his formal education at Séminaire de Québec before entering into business.
Charles had a varied career in business, encountering financial difficulties on at least three occasions, one of which ended in bankruptcy.
He became an investor in the north shore railway and specifically the towing service and tugs.
Baby entered politics in 1861, being elected as a member of the Legislative Council of the Province of Canada, and serving until his death.
The years of government contracts and behind the scenes deals had made him a man of great influence in Quebec.