The traditional story is that the family was founded by Pazzo di Ranieri, first man over the walls during the Siege of Jerusalem of 1099, during the First Crusade, who returned to Florence with flints supposedly from the Holy Sepulchre, which were kept at Santi Apostoli and used on Holy Saturday to re-kindle fire in the city.

[1][2]: 131  The historical basis of this legend has been in question since the work of Luigi Passerini Orsini de' Rilli [it] in the mid-nineteenth century.

[1] The first apparently historical figure in the family is the Jacopo de' Pazzi il Vecchio [it] who was a captain of the Florentine (Guelph) cavalry at the battle of Montaperti on 4 September 1260, and whose hand was treacherously severed by Bocca degli Abati [it], causing the standard to fall.

[8]: 149 Early in 1477, Francesco de' Pazzi, manager in Rome of the Pazzi bank, plotted with Girolamo Riario, nephew and protégé of the pope, Sixtus IV, and with Francesco Salviati, whom Sixtus had made archbishop of Pisa, to assassinate Lorenzo de' Medici and his brother Giuliano to oust the Medici family as rulers of Florence.

[2]: 138  Most of the conspirators were soon caught and summarily executed; five, including Francesco de' Pazzi, were hanged from the windows of the Palazzo della Signoria.

[2]: 140  Jacopo de' Pazzi, head of the family, escaped from Florence, but was caught and brought back.

From there it was thrown into the Arno; children fished it out and hung it from a willow tree, flogged it, and then threw it back into the river.

Guglielmo de' Pazzi, husband of Lorenzo's sister Bianca, was placed under house arrest,[2]: 141  and later forbidden to enter the city; he went to live at Torre a Decima, near Pontassieve.

[10][a] Construction began in 1442 in a cloister of the church, and continued after the death of the patron in 1445 and the architect in 1446; work was interrupted by the Pazzi plot and the chapel was never completed.