This apparent contradiction arises because many asteroids have highly inclined orbits, so although they may have a perihelion less than that of Earth, their paths can never cross.
Specifically, an asteroid is a PHA if its Earth minimum orbital intersection distance (MOID) is <0.05 AU and its absolute magnitude is 22 or brighter.
[2] The concept of PHA is intended to replace the now abandoned strict definition of ECA (Earth-crossing asteroid) which existed for a few years.
Determining if an asteroid was an ECA required calculation of its orbits millennia into the future, including planetary gravitational perturbations, to assess whether a collision with Earth was possible and this has proved to be impractical.
On the other hand, small gravitational perturbations of the asteroid around its orbit from planets that it passes can significantly alter its path.