He was born in Nîmes and may have been the brother of Leontius of Fréjus.

Castor was a lawyer and married to a wealthy widow.

Castor founded the monastery of Manauque (Monanque) in Provence which followed the monastic rule of John Cassian.

John Cassian wrote the De institutis coenobiorum at the request of Castor.

His relics are still preserved in the cathedral of Apt, of which he is one of the patrons.