Both of his parents died when he was very young, so at the age of two years and ten months he was taken in by neighbors, Samuel and Fanny Parrish, who raised him to adulthood.

In the late 1830s, as Latter Day Saints began gathering in nearby Nauvoo, the Parishes and Smith became interested in Mormonism.

During his time as bishop, Smith was involved in the Mormon Reformation, and accompanied Jedediah M. Grant in a tour of Utah in which the merits of rebaptism were presented.

He was elected to finish the unexpired term of Charles C. Rich, who had resigned so that he could travel to Europe as a missionary for the LDS Church.

In 1877, Smith became the first president of the newly organized Davis Stake of the LDS Church, and he served in this position until his death.

In 1885, Smith and two other men traveled to western Canada to examine the possibility of establishing Mormon colonies in the area.