Gulski spent two years in hiding and eventually fled in 1875 via Antwerp to the United States, where he was offered a pastorship in Berlin, Wisconsin.

A location was chosen on West Becher and South 15th Streets to construct the new church with the support of Archbishop Michael Heiss.

Gulski was a gifted orator, and many of Milwaukee's south side residents preferred to attend mass at St. Hyacinth because of his beautiful homilies.

Other Polish parishes in Milwaukee offered shelter to orphans, as well as to young ladies with unplanned pregnancies who were often rejected by their families.

It was felt that a Polish orphanage was necessary to preserve the ethnic heritage of the children so that they would not be lost to the American cultural "melting pot".

Gulski presented the city of Milwaukee with a large statue of Tadeusz Kościuszko for Kosciusko Park that was paid for by private donations.

Many believed that by having been forced to leave his beloved St. Hyacinth parish, which he had built from the ground up, Gulski died of a broken heart.