Running on a platform of economic growth via reconciliation between whites and freedmen, Miller often was opposed by members of his own party during the infancy of the Lost Cause ideology.

Miller's father was a farmer and register of the United States General Land Office active in Democratic politics, including serving as a presidential elector twice.

[1] At the age of thirteen, Miller is said to have publicly challenged notable local and fervent Whig Fent Noland regarding Martin Van Buren's credentials.

Although discouraged from pursuing the legal profession by his father, Miller moved from the family farm to Batesville to read law after turning twenty-one in 1844.

[Note 1] He was elected Independence County Clerk in 1848, and married Susan Elizabeth Bevens, the daughter of Third District Judge William Bevins, the following year.