Born in Washington, D.C., he attended the Lawrenceville School, Phillips Academy in Andover, Massachusetts and Harvard University.

During the Spanish–American War he enlisted as a private in the First West Virginia Volunteer Infantry, becoming assistant adjutant general in 1898.

Elkins was an industrialist with interests in railroads, banking, utilities, and coal mining; he was appointed as a Republican to the U.S. Senate to fill the vacancy caused by the death of his father, Stephen Benton Elkins, and served from January 9 to January 31, 1911, when a successor was elected.

During World War I he served as a major with the 7th Division of the United States Army in France, 1917–1918.

His sister Katherine Hallie "Kitty" Elkins (Jan. 14, 1886 – Sept. 3, 1936) was engaged for some time to Prince Luigi Amedeo, Duke of the Abruzzi (1873–1933), a cousin of the king of Italy.