[1] He possessed great organising ability and an attractive personality and was described by Edward White Benson, Archbishop of Canterbury, as "the most beautiful combination of enthusiasm, manliness and modesty.

"[1] Atlay married in 1859 Frances Turner, younger daughter of William Martin, a major of the Bengal Army.

Atlay died on 24 December 1894 aged 77 and is buried in Hereford Cathedral where he has a magnificent memorial in the north transept,[1] the work of James Forsyth.

[5] On the completion of his episcopate, Atlay was presented with a picture of himself by Edward Arthur Fellowes Prynne from the Diocese of Hereford.

Both were in the possession of Mrs Atlay, and replicas of the latter were hung in the Bishop's Palace at Hereford, and in the combination room at St John's College, Cambridge.