He was the second son of Col. Baldwin Fulford (1775–1847) of Great Fulford in the parish of Dunsford, Devonshire, lord of the manor of Dunsford and an officer in the Inniskillen Dragoons and Lieutenant-Colonel of the Devon Militia,[1] by his wife Anna Maria Adams, eldest daughter of William Adams (1752–1811), MP for Totnes, of Bowden House, Ashprington, near Totnes.

The Duke of Rutland instituted him to the rectory of Trowbridge, Wiltshire, in 1832, where he lived for ten years, and as a justice of the peace as well as a clergyman commanded respect and conciliated goodwill.

In 1842 he accepted the rectory of Croydon, Cambridgeshire, which he held until 1845, when he was nominated by Earl Howe as minister of the Curzon Chapel in Mayfair, London.

On 20 January 1852 the primary visitation was held, when he won great respect from all parties by his declaration that the church of England in Canada, politically considered, "exists but as one of many religious bodies."

The bishop cheerfully co-operated with all the societies that were established for benevolent, scientific, and philanthropic purposes, and wrote papers for, and delivered lectures at, mechanics' institutes and working men's clubs.

Unfortunately the great cost of this building involved the diocese in a heavy debt, the thought of which so preyed on the bishop's mind that he practised the utmost economy throughout the remaining years of his life in an endeavour to pay off the amount.