At his death he was general director and secretary of the Public Library, Museum, and Art Gallery of South Australia.
Three years later he returned to the city, where he was employed as accountant and cashier for the firm of Herford and Boucaut, living at Norwood.
On 1 June 1859 he has appointed secretary to the Board of Governors of the South Australian Institute (which then consisted of (later Sir) Samuel Davenport, the Rev.
Kay was a member of the Unitarian Church, Wakefield Street, and when younger was involved in the South Australian Militia.
In 1853 he married Ann "Annie" Catcheside (1826 – 9 May 1886); they had five daughters, notable for the influential Misses Kay's School,[3] which they conducted at the family home, 10 Trinity Street, College Town c. 1888 – December 1904.