His mother died when he was four and Caresser was brought up by his grandmother and aunt; his unrecorded 1945 calypso "I ain’t got no papa" suggests that he had no knowledge of his father.
[1][better source needed] Caresser moved to Port-au-Prince in Haiti in the late 1920s or early 1930s, where, according to a story he later told, a witch doctor gave him a ring decorated with the image of a snake.
The ring was supposed to have the power to attract crowds, but Caresser threw it away when he noticed the snake moving as he passed by a church.
Performing in the capital, Georgetown, as part of a vaudeville show that included dancers, comedians, and impersonators, Caresser sang his recently recorded calypsos "Madam Khan" and "The More They Try to Do Me Bad (The Better I Live in Trinidad)".
[citation needed] During his stay in Georgetown, Caresser began a romantic relationship with the famous dancer Peggy Daniels, later known as "The Rhumba Queen".
[2] In 1947, Caresser moved to Canada after securing a 5-year contract with the Canadian Broadcasting Corporation (CBC) to record a regular radio show.
[4][better source needed] Caresser began a relationship with Monique Côté from Abitibi in Northern Quebec, and their first son, Camille, was born in Montreal in 1953.