[3] He trained for ordination at Ridley Hall, Cambridge,[4] and graduated with a Master of Arts (MA) degree in pastoral theology in 1998.
[5] Rumsey was ordained in the Church of England: made deacon at Petertide 1997 (26 June) by Richard Chartres, Bishop of London, at St Paul's Cathedral;[6] and ordained priest the Petertide following (5 July 1998), by Graham Dow, Bishop of Willesden, at St Martin's Church, Ruislip.
In an interview following his consecration as Bishop of Ramsbury, Rumsey stated that Evensong is his favourite form of church service.
[13] In November 2023, he was one of 44 Church of England bishops who signed an open letter supporting the use of the Prayers of Love and Faith (i.e. blessings for same-sex couples) and called for "Guidance being issued without delay that includes the removal of all restrictions on clergy entering same-sex civil marriages, and on bishops ordaining and licensing such clergy".
[14] Also in 2023, Rumsey took part in a ceremony to bless the River Marden as a sentient being, together with Kate Dineen, a shamanic practitioner.