During the American Prohibition banning the sale of alcohol, Abe Lerner used his metalworking abilities on the side to make stills for Cleveland gangsters and bootleggers, and used his seven-year-old son Al as a courier for payments.

It was a rough neighborhood with regular mob wars between rival gangsters, multiple killings on Lerner's street, and payoffs to the local police.

[5] As a child, Lerner helped to bring in money by climbing onto tables in local saloons and singing songs such as "All Alone by the Telephone", and then collecting coins thrown by the patrons.

When he saw a performance by Bill Robinson at the Palace Theatre, he decided he wanted to learn tap dancing, so studied with Roy Lewis, and soon was winning amateur contests.

His skills continued to improve, and by the age of 17 he had worked his way up to earning $15/week, playing at clubs such as Shadowland, and developing an affinity for jazz by listening to records by Earl Hines.

It was during this time that Lerner began composing, such as writing an instrumental with Harry James, "Music Makers", which became the band's theme.

When the James Band finally broke up in 1944, Lerner stayed in Los Angeles and joined with singer Dick Haymes, with whom he worked for the next thirteen years as musical director.

Over the course of his career, Lerner worked with many artists from the Big Band era of music, such as Charlie Barnet, Tommy Dorsey, Glenn Miller, Rosemary Clooney, Kay Starr, and Pat Boone.