John grew up near Quebec City on the family estate which was one of the properties his father, Henry Caldwell, had obtained from Governor James Murray by way of a 99-year lease.
He never practiced law as he was administering his father's assets and ongoing business affairs.
In 1800, he ran for, and was elected in the riding of Dorchester along with Jean-Thomas Taschereau for the 3rd Parliament of Lower Canada.
Caldwell was immediately appointed to the Legislative Council of Lower Canada where he served until his death.
Caldwell died at the Tremont House in Boston, Massachusetts in 1842 and is buried in the hotel's lot at Mount Auburn Cemetery.