Born on January 31, 1869, in Harrisburg Pennsylvania, Small studied botany at Franklin & Marshall College and Columbia University.

It was mainly at that time of his life when he issued several exsiccatae, among others the series Mosses of the Southern United States, distributed from the Herbarium of Columbia College by John K. Small (1897).

Small's doctoral dissertation, published as Flora of the Southeastern United States in 1903, and revised in 1913 and 1933, is a detailed floristic reference for much of the South.

[3] Assisted by the patronage of Charles Deering, Small traveled extensively around Florida recording plants and land formations.

[4] Small was an early botanical explorer of Florida, documenting many species for the first time, although the flora and fauna were well known to the local Seminole Indians.