The youngest of nine children of Thomas and Ann Kay of Knaresborough, he was born on the 8th of April 1820, at Pickering, North Yorkshire.
He passed two years at Giggleswick school, and, together with James Fraser, gained an open scholarship at Lincoln College, Oxford, on 15 March 1836.
[1] In 1865 Kay was made select preacher before the university, and in 1866 was presented by his college to the rectory of Great Leighs, Essex, where he remained for the rest of his life.
[1] In Calcutta Kay published several pieces at the college press, including his translation of the Psalms with notes, 1864 (3rd edit., enlarged and improved, London, 1877).
One of the Old Testament revisers in 1870, he was notably conservative in his criticism, and contributed to the Speaker's Bible commentaries on the Book of Isaiah (1875) and the Epistle to the Hebrews (1881).