His parents, Henry and Mary Ann (née Amyss) Milnes joined the LDS Church and left England with their family in 1854.

They stayed in Newton, Iowa, for the winter of 1859–1860, and in 1861 arrived in Coldwater, Michigan, after traveling the entire distance by oxen-drawn wagon.

With the outbreak of the Civil War, Milnes enlisted as a private in Company C, Seventeenth Regiment, Michigan Volunteer Infantry, June 30, 1862.

The forces mustered at Detroit, and left the state on August 27, 1862, going straight to Washington, D.C. On September 14, he saw action in the Battle of South Mountain in Maryland.

The regiment was discontinued as a military unit and the survivors were taken to the headquarters of General Orlando B. Willcox, where they served guard duty until the end of the war.

The partnership prospered for ten years, after which Milnes sold hist interest and started his own business.

Milnes was chosen from a special election in April 1895 as a Republican from Michigan's 3rd congressional district to the 54th Congress to fill the vacancy caused by the resignation of Julius C. Burrows.