He was born on 30 May 1797 in Westport, Ireland, where his father held a curacy, the only son of Walter Shirley, by his wife Alicia, daughter of Sir Edward Newenham.
In 1822 he was appointed assistant lecturer of Ashbourne and curate of Atlow and was awarded the prize for the English essay at Oxford, the subject being the Study of Moral Evidence.
[1] He acted as chaplain at Rome in the winter of 1826–7, and during his residence there he became intimately acquainted with Christian Karl Josias von Bunsen and Thomas Erskine, as well as with Charles Lock Eastlake and David Wilkie.
In the autumn of 1827 he was married at Paris to Maria, daughter of William Waddington, and at the same time, his father resigned the living of St Michael's Church, Shirley in his favour.
In 1829 he alienated some of his friends by his outspoken advocacy of Catholic emancipation; in later years he estranged others by refusing to support measures against the Tractarians.
In November 1846 he was appointed Bishop of Sodor and Man by Lord John Russell; but because of a serious illness, he was not consecrated until 10 January 1847.