Budge was born in Lanark, Scotland and moved in his teen years to Glasgow for employment.

Among other assignments, in 1855 the president of the European Mission Franklin D. Richards sent Budge to Saxony to respond to inquires about the LDS Church from Karl G. Maeser.

He was the captain of a company of seventy-two wagons from Winter Quarters to Salt Lake City.

Budge was an active public servant, and was elected as the local superintendent of schools in 1872.

After losing a race for the Idaho Territorial Council from Bear Lake County in 1874, he won elections to the same position as a Democrat in 1876 and 1880.