The origin of the name comes from the explorations of the bandeirante Fernão Dias Paes Leme who was given instructions to pass over a river four times, thus "Passa Quatro".

Mountains include the fourth tallest peak in Brazil, the Pedra da Mina (2,798 m) located within Serra Fina and the point of Serra da Mantiqueira where the boundary of São Paulo is located.

The Pico dos Três Estados is another landmark located on the boundaries of São Paulo, Minas Gerais and Rio de Janeiro.

There are dozens of other peaks over 2,000 m. The city was declared a hydro-mineral spa in 1941 due to its various fountains of torio-radioactive waters, which are reputedly sought out for curing liver and kidney problems.

A tourist train service (narrow gauge) was inaugurated in the beginning of 2004 between the cities of Passa Quatro and Manacá.