[2] Ballard received a recess appointment from President Abraham Lincoln on October 16, 1861, to a seat on the United States District Court for the District of Kentucky vacated by Judge Thomas Bell Monroe,[1] who had resigned to take a seat in the Congress of the Confederate States.

[1] He was confirmed by the United States Senate on January 22, 1862, and received his commission the same day.

[2] "His district was responsible for more indictments for treason and conspiracy than perhaps any other" and he was "regarded as fair-minded and guided by the law, not prejudice.

[2] Ballard oversaw the trial and conviction of two White men who slaughtered and mutilated a family of African Americans before it was appealed to the United States Supreme Court in Bylew v. United States.

[5] In addition to his duties as district judge, Ballard served as president of the Kentucky National Bank and the Cave Hill Cemetery Company and was active in various civic organizations.