He had a difficult childhood, being partially raised in an orphanage when his father was forced to leave the family to look for work and his mother could not feed her children.
[1] The fifteen-year-old Fisher was recruited by Fulton to work as an apprentice draughtsman for $2 per day in his Uniontown, Pennsylvania office in the fall of 1916.
At night and on weekends he studied Beaux-Arts courses at the Beaux Arts Institute of Design in New York City.
In 1922 he and a colleague, Charles Hines, started their own architectural office in Hagerstown, Maryland, but had to close their company after only a year.
As a result he began working for the Austin company[clarification needed] and Conover Engineering, supervising the conversion of Detroit's factories for wartime production.