Among his more than 200 publications were translations of the works of Archimedes (1880 and 1912), Euclid (with Heinrich Menge) (1883–1916), Apollonius of Perga (1891–93), Serenus of Antinouplis (1896), Ptolemy (1898/1903), and Hero of Alexandria (1899).
Heiberg inspected the vellum manuscript in the library of The Church of the Holy Sepulchre at Istanbul in 1906, and realized that it contained mathematical works by Archimedes that were unknown to scholars at the time.
Heiberg was permitted by the Greek Orthodox Church to take photographs of the palimpsest's pages, and from these he produced transcriptions, published between 1910 and 1915 in a complete works of Archimedes.
Heiberg's examination of the manuscript was with the naked eye only, while modern analysis of the texts has employed X-ray and ultraviolet light.
Heiberg is one of the men seen in Peder Severin Krøyer's monumental 1897 group portrait painting A Meeting in the Royal Danish Academy of Sciences and Letters.