His father, David Fisher (1729–1782), a craftsman, and his mother Mary, née West (1730–1819) lived in Norwich.

As he was handsome, and had a fine tenor voice, he attracted the attention of Giles Barrett, the manager of the Norwich Theatre Royal.

Initially the company was in competition with several other troupes, but Fisher took a more business like approach (for example paying bills promptly) and taking care of his staff of twenty or more (most of them members of the Fisher or Scraggs families but with a leaven of other professional actors), and had better equipment such as scenery (which most other touring theatres in the region did not have), and a much larger collection of good quality costumes, so gradually the competition withered away, which left north East Anglia his.

[1] Fisher catered to all classes but his decisions as to where to visit and how long to stay were made on the number of gentry and noble seats in a region, as they were the ones who would pay a premium price to be entertained.

Over the next sixteen years, usually raising capital with share offerings to local patrons, he opened theatres in Halesworth (1812), Woodbridge (1814), Eye (1815), East Dereham (1816), Sudbury (1817), Thetford (1818), Beccles (1819), Swaffham (1822), Newmarket (1825), Bungay (1828), and North Walsham (1828).

They had five children, of whom their daughter Elizabeth and youngest son Henry died young.