He was a son of Lerwick, whose grandfather had left Dunrossness as a soldier during the Napoleonic period and lived in Edinburgh for a time before settling in 'da toon' as a shopkeeper.
In his lifetime he was best known for a novel entitled The Viking Path – A Tale of The White Christ (1894), set in Shetland and Norway at the coming of Christianity.
The best-known of his work now is probably his verse in dialect, as published in Rasmie's Büddie: poems in the Shetlandic (1891), which was republished in 1913, and later in 1979 with illustrations by Frank Walterson.
Burgess brings out the darker side of this community, the hypocrisy and occasional dishonesty, the servility of some crofters towards the gentry, the malicious gossip, the petty jealousies and spitefulness within the congregation and the damage done by itinerant hot-gospellers and their infernal, illegitimate-producing revival meetings' as Hakki, the agnostic schoolmaster, calls them.
It is certainly an unusual novel for its time when the Scottish Kailyard school of writing was still at its height ..." A street in Lerwick, Haldane Burgess Crescent, is named after him.