He was the older of two children born to Nancy (née O'Neill) and Walter Grimes; his father worked as a fitter and turner with New South Wales Railways.

He was educated at Albury High School for three years, but after his mother's death from bowel cancer he and his sister were sent to live with an aunt in Sydney.

Grimes completed his education at Fort Street High School and went on to study medicine at the University of Sydney (MBBS 1962).

After returning to Australia he served as district medical officer at Cygnet and then opened a general practice in the Launceston suburb of Riverside.

[3] In March 1980, Grimes was suspended from parliament for 24 hours for using the words "pimp" and "fascist" and accusing Prime Minister Malcolm Fraser of "setting up" a demonstration outside a nursing home at which nine people were arrested.