In the past it was an important urban center of northern Lesser Poland, with town charter granted to Skaryszew as early as 1264.

First documented mention of Skaryszew comes from the year 1198, when the village belonged to the Abbey of Order of the Holy Sepulchre, located at Miechów.

Due to the efforts of the Order of the Holy Sepulchre, merchants and skilled artisans were attracted to the quickly growing village.

Skaryszew was destroyed in the Mongol invasion of Poland, and soon afterwards, Duke of Kraków and Sandomierz Bolesław V the Chaste granted the village the so-called Środa Śląska town charter, based on the charter of Nowy Korczyn (see also Magdeburg rights).

Following the Austro-Polish War of 1809 it was regained by Poles and included within the short-lived Duchy of Warsaw, and after its dissolution it belonged to Russian-controlled Congress Poland (since 1815).

The war and German occupation brought destruction to the town, and its Jewish residents were murdered in the Holocaust.

In 1633, King Władysław IV Vasa allowed the markets to be organized on the first Monday after Ash Wednesday, which usually is in mid-March.