Herbert was educated at St Andrew's University and Trinity College, Cambridge: he was admitted pensioner on 15 May 1860 and matriculated that Michaelmas; graduated Bachelor of Arts (BA) 1864 and proceeded Master of Arts (Cambridge) (MA Cantab) 1867.
[6] He began his ordained ministry as vice-principal (1865—67) and then the principal (1867–68) of Carmarthen Training College;[2][7] he then became Curate of Newland, Gloucestershire until 1870.
[4] He made his first return to England in 1874,[2] becoming the rector of All Saints' Church, Huntingdon (1874–77), then Vicar of Loddington, Leicestershire and Chaplain of Launde Abbey[1] (1877–93) and rural dean of Gartree (1886–89).
[8][4] From 1891 to 1897 he was Provost of St Andrew's Cathedral, Inverness[9] when he was appointed to the episcopate[10] as the fourth Bishop of Antigua.
[11] He was consecrated into bishop's orders on 18 July 1897 at Lambeth Palace Chapel, by Frederick Temple, Archbishop of Canterbury.