Johnson began playing professionally as Florence Mills' accompanist, and formed his own band in 1924.
He briefly worked with Henri Saparo and Noble Sissle, and then he joined Sam Woodings band and traveled to Europe in June 1928.
In the mid 1930s he made some recordings with the Quintette du Hot Club de France.
While living in Amsterdam, he co-lead a band with Lex Van Spall, and they played regularly at the Negro Palace in a trio with Coleman Hawkins.
He later worked at the Negro Palace, and then with Max Woiski in a club called La Cubana, in Amsterdam.