He was born in Glasgow, but moved to the Isle of Lewis at the age of two, where he and his two brothers were brought up by their widowed mother in the small crofting town of Bayble, which also produced Derick Thomson.

Crichton Smith was brought up in a Gaelic-speaking community, learning English as a second language once he attended school.

Friend and poet Edwin Morgan notes that unlike his contemporaries (such as Sorley Maclean and Derick Thomson), Crichton Smith was more prolific in English than in Gaelic, perhaps viewing his writing in what, from Crichton Smith's view, was an imposed non-native language as a challenge to English and American poets.

Crichton Smith's work also reflects his dislike of dogma and authority, influenced by his upbringing in a close-knit, island Presbyterian community, as well as his political and emotional thoughts and views of Scotland and the Highlands.

A number of his poems explore the subject of the Highland Clearances, and his best-known novel Consider the Lilies (1968) is an account of the eviction of an elderly woman during such times.