Her father died, when Roy was ten years old, and she was raised by her maternal uncle Soumendranath Tagore in Calcutta.
She also recalled witnessing her uncle being arrested by British police and forced on to a train at Calcutta railway station.
After returning to the United States, Lois Kellogg wrote a letter to Ena Tagore, requesting her to enroll her daughter at Wellesley College.
Riboud would later describe her as "one of three absolutely adorable, beautiful young Indian girls who were visiting from Wellesley College".
After their first meeting, Riboud invited Roy to take a walk during which he asked about her uncles, and they discussed India, her philosophy studies, and modern art.
Roy agreed, and in the summer of 1949, the three traveled by car to Seattle, along the Pacific Coast to Arizona, and across the country to the home of American poet and critic Charles Olson at Black Mountain College.
It is reported that the couple had an extensive friendship circle which included political figures like François Mitterrand, Indira Gandhi and Ne Win and art personalities, such as Yves Tanguy, Henri Cartier-Bresson, Isamu Noguchi, M. F. Hussain, Joan Miró and Max Ernst.