Born in Mulloloughan, County Monaghan, Ireland, Kelly was schooled in Dublin, and worked as a teacher and law clerk before emigrating to PEI with his wife, Catherine Lennon, in May, 1835, ten years before the Great Famine.
Kelly was elected as the running mate of the quick-tempered Liberal George Coles, who had recently become the first Premier of Prince Edward Island.
Kelly maintained his seat (with the exception of the 1872 session, when the province's government fell over a bankrupting railway policy) until his death in 1879.
Kelly was also a Justice of the Peace, Postmaster, Taker of Affidavits for the Supreme Court, Commissioner for Taking Recognizance of Bail, member of the Executive Council, Land Survyer and Commissioner of Crown Lands, member of the Board of Education, and captain of the Fort Augustus Rifles (the local militia).
Both politicians also voted for the "better terms" resolution in 1873, by which time Confederation had come to be viewed both as an economic necessity, and as a benefit for Prince Edward Island's Catholics.