Dexter was born at Melbourne, Derbyshire, England, and became an apprentice at the Derby China factory painting flowers and birds in the Chinese and Japanese styles.
Dexter then studied at Paris, and returning to England, married Caroline née Harper at Nottingham in 1843.
Dexter was at Bendigo in August 1853, where William Howitt heard him advocating republican doctrines at a meeting of diggers.
His wife came out from England at the end of 1854, and in March 1855 they together opened a gallery of arts and school of design in Bathurst Street, Sydney.
This apparently was not a success for they went to live at Stratford, Victoria, in 1856, and there made the acquaintance of Angus McMillan.