Bursche studied Lutheran divinity at the University of Tartu and became a member of the "Konwent Polonia", a Polish student fraternity, established in 1828.
Bursche started working as a vicar in Warsaw in 1884 and married Amalie Helena Krusche in 1885.
In 1905, he instituted the use of the Polish language in Lutheran church services, which had previously been only ministered in German.
After the outbreak of the First World War in 1914, the Russian administration of Congress Poland began to deport members of the Lutheran church, considering them as Germans.
Bursche's son Stefan was killed by the Gestapo in 1940, his daughter Helena, director of the Lutheran Anna – Wasa Lyceum in Warsaw, died in 1975, his daughter Aniela, journalist at the Lutheran newspaper Zwiestun, died in 1980 in Warsaw.