At the age of ten years he was sent to Newport, New Hampshire, to attend a preparatory school before entering Dartmouth College.
Knowing that if discovered by his parents he would be obliged to return to the preparatory school, he did not communicate with them for a period of five years, during which time he remained steadily at his post at the Advertiser, learning all the details of the printing business.
On the defeat of the Lecompton Constitution by Congress, Kelley sold out his interest in the Squatter Sovereign and moved to Doniphan County, Kansas, where he married Miss Mary L. Foreman, and where his first child (Mrs. Kate Napton) was born.
On returning from a campaign in Mississippi, the regiment was reorganized, and Kelley was assigned to the recruiting service, after which his connection with the Confederate army terminated.
In 1866 he went to Bear Gulch, where he remained a short time, after which he moved to Deer Lodge, where he and his family finally settled.
He was one of the original incorporators of the Rock Creek Ditch Co., organized in an early day, and still at the time of his death had an interest in that company.
He was at the time of his death largely interested in the Southern Cross and other mining claims in the neighborhood of Cable, Montana.
In 1885 Kelley was appointed by President Grover Cleveland the 5th United States Marshal for Montana, and served in that office with official integrity until the day President Benjamin Harrison was inaugurated, when he resigned, believing that the party in power should have control of all the Federal patronage and be held responsible for it.