Ballard was born on 22 April 1791 in Islington, Middlesex, the son of Edward Ballard, an alderman of Salisbury, Wiltshire and Elizabeth, daughter of G. F. Benson of that city.
He obtained a situation in the Stamp Office in 1809, and, having resigned this appointment, entered the Excise Office, which he also left of his own accord in 1817 in which year he became a contributor to Woollers' Reasoner.
Robert Benson, his cousin, and Henry Hatcher received no small help from him in writing their History of Salisbury (1843), which formed part of Hoare's Wiltshire.
He helped John Gough Nichols in the works undertaken for the Camden Society.
In 1848 he brought out some parts of a continuation of John Strype's Ecclesiastical Annals in a publication called the Surplice, but this paper and Ballard's scheme soon came to an end.