Southard was born in Hempstead, Long Island, New York.
He moved with his parents to the Basking Ridge section of Bernards Township, New Jersey in 1755, where he attended the common schools and worked on a farm.
Later, Southard served as a private and later as wagon master during the American Revolution.
After the war, he engaged in agriculture, before serving as a Justice of the Peace from 1787 to 1792.
During the 11th Congress, Southard was Chairman of the Committee on Revisal and Unfinished Business.