Most of these early Jews were merchants and traders, having moved to Mobile after the French lost their North American possessions to Great Britain in the Treaty of Paris.

Jews were not allowed to officially reside in colonial French Louisiana due to the infamous Code Noir, a decree passed by France's King Louis XIV in 1685.

Although no records survive that document the spiritual activities of Mobile's earliest Jewish citizens, on June 22, 1841 Congregation Sha'arai Shomayim purchased plots in the city's Magnolia Cemetery.

[3]: 1–7 Despite the earlier purchase of burial plots, the congregation did not become officially organized until January 25, 1844 when a constitution and bylaws were registered with the Mobile County Probate Court.

The congregation's full name in the document is listed as Sha’arai Shomayim U - Maskil El Dol or Gates of Heaven and Society of Friends of the Needy.

[3]: 1–7 During 1844 and 1845, Congregation Sha'arai Shomayim held services on Friday evenings and Shabbat morning in private homes.

In late 1848, Benjamin da Silva moved to New Orleans and Baruch M. Emanuel was appointed to lead the congregation.

[3]: 7–39 With the coming of the American Civil War, Congregation Sha'arai Shomayim saw many of its members serving in the Confederate States Army.

Thousands of Mobilians took part in his burial service and his marker is inscribed with a letter that he wrote to his parents the night before the attack.

[5] Plans for a new and larger synagogue began to take shape in the first decade of the 20th century, with the cornerstone for the new temple laid on June 6, 1906 at the corner of Government and Warren Streets.

A building fund of $25,400 had been accumulated, but the congregation president, Jacob Pollock, estimated that $5000 more was needed to complete a structure that would "honor the Jewish population of Mobile.

The bold design was built like a fortress, with two large corner towers that framed an entrance, recessed into a deep and wide low arch.

Eight rabbis took part in the dedication processional: Maximilian Heller of New Orleans; Henry Ettleson of Fort Wayne, Indiana; Bernard Ehrenreich of Montgomery, Alabama; Jacob Schwarz of Pensacola, Florida; David Marx of Atlanta, Georgia; Max Raisin of Meridian, Mississippi; Morris Newfield of Birmingham, Alabama, and Alfred Geiger Moses.

On July 18, 1952, the committee announced that it was buying a lot on Springhill Avenue for $40,000 and hiring architect T. Cooper Van Antwerp to design a new synagogue in the Modernist style, with a budget of $250,000.