Born Raphael Silverman in Ithaca, New York, his career included playing in the Boston Symphony Orchestra and co-founding the Juilliard String Quartet.
At the age of 16 Hillyer attended the Curtis Institute of Music, followed by studies at Dartmouth College, from which he graduated Phi Beta Kappa with a mathematics degree in 1936.
He then completed graduate work in music under Walter Piston and Hugo Leichtentritt at Harvard University, where he played frequent recitals with his friend and classmate, Leonard Bernstein.
With a borrowed viola and an intensity for which he was becoming well known, Hillyer played the audition and was chosen to be the violist and founding member of what became the Juilliard String Quartet.
After retiring from the Juilliard String Quartet in 1969, Hillyer performed frequently as soloist and collaborator with other chamber music groups.