Grimes also had a fruitful long-term working relationship with Sydney Pollack, making seven films with him, and he was also employed by directors such as David Lean, Peter Yates, Mark Rydell and Ulu Grosbard.
One of his great strengths as a production designer was his skill as an artist: in researching and preparing a film he would produce many beautiful, dramatic and atmospheric sketches.
Grimes grew up in Haslemere, Surrey, Peldon in Essex, and Lansdowne Road, Notting Hill Gate, London.
He went to St Martin's School of Art in the late 1940s, where he met and then married Kathleen Grimes (née Sanders).
Carmen Dillon took him 'under her wing' and he also worked alongside Oliver Messel, Vertchinsky, Paul Sheriff, Hein Heckroth, Ivor Beddoes, John Box and Ralph Brinton.