The family moved to Waterford in 1847, where Packard learned the carpenter's trade.

At age 15 he left home, working for engineers and architects in Lowell and Lawrence, Massachusetts.

[2] In 1866 he moved back to Vermont, having been employed as a foreman with E. & T. Fairbanks & Co. in St. Johnsbury.

He developed a friendship with Franklin Fairbanks, who had been a partner in the firm since 1856, and served as its president from 1881 to 1895.

Through the beneficence of the Fairbanks family, Packard designed most of the town's major buildings.