Born in Norwich, Norfolk (England), 27 July 1789, he was the second son of Jacob Mountain (1749–1825), a bishop and politician, by his wife Elizabeth Mildred Wale[1] co-heiress of Little Bardfield Hall, near Thaxted, Essex.

He lived with his family at Marchmont House, near Quebec, where he received his early education before returning to England at the age of sixteen to study under private tutors until he matriculated from Trinity College, Cambridge, graduating Bachelor of Arts (BA) in 1810, and Doctor of Divinity (DD) in 1819.

Stewart shortly afterwards proceeded to Britain (dying in 1837), and the care of the entire diocese was under Mountain's administration (remaining Bishop of Montreal) until 1839, when Upper Canada was made a separate see in Toronto .

During the greater part of his ministerial career he had to perform long, tedious, and often dangerous journeys into the interior of a wild and unsettled country, paying frequent visits to the north-west territory, the eastern townships, the Magdalen Islands, and the shores of Labrador; also to Rupert's Land, some 3,600 miles, in an Indigenous canoe.

He came to Britain in 1853 to confer with William Broughton, the metropolitan of Australasia, on the subject of synodical action in colonial churches, and he received the degree of Doctor of Civil Law (DCL) at the University of Oxford.