His parents initially intended him to pursue a military career, but Castillon gave up plans for professional soldiering in favour of music, which he learned first in his birthplace and then in Paris, where he studied piano and composition, initially with Charles Delioux.

For composition, he attended the classes of Victor Massé and, from 1869, of César Franck at the Paris Conservatoire.

Before his studies with Franck, he had completed his Opus 1, a piano quintet, probably inspired by the earlier example of his friend Camille Saint-Saëns.

In fragile health at the best of times (volunteering during the War of 1870, he fell ill and was demobilized in 1871), he died of complications from fever in 1873 at age 34.

He was also active in Parisian musical life, in particular helping to create, in 1871, the Société Nationale de Musique, of which he was the first secretary.