He attended Sidwell Friends School of Washington, D.C., and graduated from Yale University in 1916.

He served as Referee in Bankruptcy from 1923 to 1941, and as police and juvenile judge for Talbot County, Maryland from 1934 to 1938.

During the Second World War, Miller served as a colonel in the Infantry in the U.S. Army from 1942 to 1946, where he saw duty in North Africa, India, and China.

Miller did not sign the 1956 Southern Manifesto and voted in favor of the Civil Rights Acts of 1957.

He unsuccessfully sought candidacy in 1962 for the United States Senate, and afterwards resumed the practice of law.