Fischer, officially a son of the Duke's councillor and first master chef, but possibly an illegitimate son of the Duke had his education at the Gymnasium Illustre in Stuttgart and then completed an apprenticeship with the sculptor Johann Christoph Friedrich Beyer and from 1760 with the painter Nicolas Guibal.

[2] Fischer participated in the construction of the Cathedral of St. Eberhard in the Königstraße in Stuttgart, the New Palace (1807), the Hohenheim palace, for which he was planning a large equestrian center,[3] the Castle Solitude, the pheasant garden in Weilimdorf and the first Bärenschlössle Castle (1768) in Stuttgart.

[4] He was also the architect of the early classical Franziska Church in Stuttgart-Birkach, that was presented to the population of Birkach by Charles Eugene in 1780.

[5][6] After Charles Eugene's death, Fischer designed the Schloss Kirchheim as a dower house for Franziska of Hohenheim.

[7] Among the buildings built by Fischer privately are the Vischer Palace and the Reichert House in Calw.