There he met Rebecca "Becky" Lang (1808–1891), and after a brief courtship they were married at Rome by her stepfather, Lemuel Mallory (1763–1851), on April 4, 1824.

Joshua and Rebecca Huckeby were the parents of nine children, three sons and six daughters: Mary Ann (1826–1864), Rachel Littell (1828–1883), Eliza Ellen (1830–1901), Elizabeth (c.1833–1852), John Lang (1835–1882), William Lamb (c. 1838–1903), Sarah Jane (1840–1852), Isabelle (1845–1909), and Thomas Jefferson (1847–1849).

From 1854 to 1856, he served as prosecuting attorney for the 3rd Indiana District Common Pleas Court, discharging his duties with marked ability, and for several years thereafter practiced his profession with vigor and success.

His knowledge of the roads, streams and by paths, the ravings and hollows, the school houses and places of public resort in Perry county was surpassed by no one.

His grandson, Thomas James de la Hunt, Jr. (1866–1933) wrote this of his grandfather, "He was a marked example of the Old School politician, violently unrelenting in many inherited prejudices, and always delighting to dwell reminiscently upon the political triumphs of those early years when–it was his favorite boast–he knew every man in Perry County, his politics, his religion, and the nighest way to his house."

In the 1860 presidential election, Huckeby's southern birth and conservatism allied him with the Constitutional Union Party, and he was placed upon the Bell and Everett ticket as elector for the First District of Indiana, to which Perry County then belonged.

His wife, Rebecca, was a distant cousin of John Henninger Reagan of Texas, Postmaster General of the Confederate States of America.

In 1864, Huckeby entertained Captain Edmund Morgan of the United States Navy paddle steamer USS Springfield at his home, the Virginia Place.

His daughter, Rachel Littell (Huckeby) Mason was president of the Ladies' Patriotic Aid Association which supplied the Quartermaster-General at Indianapolis with much needed clothing for soldiers.

His youngest daughter, Isabelle Huckeby, married Major Thomas James de la Hunt, Sr. (1835–1872) after the Civil War.

His grandson, Thomas James de la Hunt, Jr. (1866–1933) was a well-known writer and newspaper columnist.