After leaving school in 1907, Scherer began an apprenticeship as a stonemason at the Schwab workshop in Lörrach.

From 1921 to 1922 he was influenced by the work of the German painters Erich Heckel, Ernst Ludwig Kirchner and Karl Schmidt-Rottluff.

Eventually, Scherer founded the artist group Rot-Blau with Albert Müller and Paul Camenisch in late 1924, later joined by Werner Neuhaus.The group received a strong public reaction when they debuted their work at an exhibition at the Basler Kunstverein in 1925.

The Dreiländermuseum in Lörrach holds 118 of Scherer’s works, including many woodcuts and the "Portrait of Otto Staiger".

In 2004 the Pfalzgalerie Kaiserslautern acquired the sculpture "Das kleine Mädchen" ("The little girl") from a Swiss art gallery for over 140,000 Euros.