One after another, his father, mother, sisters and nephews died of starvation or exhaustion, as they were held in a remote labor camp in rural Cambodia.

Rithy's family suffered under the regime of Democratic Kampuchea, and after he saw his parents, siblings and other relatives die of overwork or malnutrition in the Cambodian genocide, he managed to escape to Thailand in 1979,[4] where he lived for a time in a refugee camp at Mairut.

His first documentary feature film, Site 2, about a family of Cambodian refugees in a camp on the Thai-Cambodian border in the 1980s, was awarded "Grand Prix du Documentaire" at the Festival of Amiens.

Rithy, along with director Ieu Pannakar, has developed the Bophana Center in Phnom Penh, Cambodia, with an aim towards preserving the country's film, photographic and audio history.

The center's namesake is the subject of one of his early docudramas, Bophana: A Cambodian Tragedy, about a young woman who was tortured and killed at S-21 prison.