He was taught by Abbé Sicard and deaf educator Jean Massieu, at the Institution Nationale des Sourds-Muets in Paris.

When he was one year old, Clerc fell from a chair into a fire, suffering a severe burn and obtained a permanent scar on the right side of his cheek.

The facial scar later provided the basis for his name sign, performed with the manual alphabet for "U" (thumb out), with the pads of the two fingers stroked twice downward on the right cheek.

The 1869 obituary in the New York Times says, Clerc came to Hartford in 1816 and became a teacher in 1817, then served more than 50 years "in the cause of deaf-mute instruction" and "his abilities, zeal, and graces of character made him always respected and loved.

Largely due to Clerc's contribution to the education of the Deaf in America several awards, buildings, funds, and other honors were named after him.