Williams entered professional baseball in 1907 with the San Antonio Black Bronchos and was an immediate star, posting records of 28-4, 15-9, 20-8, 20-2, and 32-8.

After the Lincolns finished an ignominious fifth (out of six teams) in the Eastern Colored League's inaugural season, Williams was released in the spring of 1924.

Williams joined the Brooklyn Royal Giants for a season, then signed with the independent Homestead Grays, where, except for a brief turn with the Detroit Wolves in 1932, he spent the rest of his career in top-level black baseball.

On August 2, 1930, at age 44, Williams struck out 27 Kansas City Monarchs in a 1–0, 12-inning, one-hit night game victory.

[7] That same year, he beat a younger Negro league star who was just bursting into superstardom, Satchel Paige, also by 1–0, in their only meeting against each other.

Among his victims were Hall of Famers Grover Cleveland Alexander, Walter Johnson, Chief Bender, Rube Marquard, and Waite Hoyt.

After joining the Homestead Grays in the late 1920s, his nickname became "Smokey Joe", and the older "Cyclone" appellation was rarely used after that.

Most modern sources lean toward Paige, but in 1952, a poll taken by the Pittsburgh Courier named Williams the greatest pitcher in Negro league history.