[3] On the recommendation of Oregon Senator George Henry Williams, Ballard was appointed territorial governor by President Andrew Johnson in April 1866.

[1] Upon his arrival, Ballard learned that the legislature of 1865 had passed an act abolishing extra pay for the governor and secretary, but retaining, and even increasing, their own and that of their clerks.

Ballard, in response, approved of the measure, and suggested that the territory be saved the whole of the extra money, including monies to Democratic legislators.

Despite this, the territorial legislature contacted federal Treasury Secretary Hugh McCulloch and advised him they were three sessions behind in pay due to the embezzlement and Lyon's disastrous administration.

In response territorial secretary Solomon R. Howlett informed McCulloch that many of the legislators had refused to sign an oath of allegiance to the union per an 1862 act of Congress and were therefore ineligible for any back pay to begin with.

Calling the legislation "presumptuous," Ballard vetoed it on the grounds that it violated the Idaho Organic Act of 1863, which established the territory.

Three subsequent appointees either declined or quickly resigned the post until Grant finally found a lasting replacement in Thomas W. Bennett, several months after Ballard left office.

They had nine children: Lonner L., Oscar, Frank Rooker, Florence E., Carrie, Maud M., Ora, and two daughters, Amanda D. and Mary J., who died in infancy.