The town, birthplace of Filippo Mazzei, lies 9 kilometres (6 mi) south of the provincial capital of Prato.

Initially, only agricultural improvements were carried out; then in 1485, work began on the villa itself, to designs by Giuliano da Sangallo, who created a large fortified, quadrangular country house built around a central courtyard.

A large central hall gave access to rooms with windows overlooking the surrounding countryside; at the time, this design was innovative.

Poggio also contains in the North East corner of the formal garden Cosimo I's tennis court (1543) intact but no longer in use.

The "tennis" of the Italian Renaissance was hugely popular at court and in the streets as in France where the sport is known as "jeu de paume".