Lord Arthur Hay was born at Yester House near Gifford, East Lothian, the son of General Sir George Hay, 8th Marquess of Tweeddale and his wife, Lady Susan Montague.

Tweeddale described about 40 species collected by Bock for the first time and was elected a Fellow of the Royal Society in 1871.

[5] The holotype specimen of the Critically Endangered and enigmatic Malabar Civet was obtained by him and gave it to Asiatic Society of Bengal in 1845.

[6] His ornithological works were published privately in 1881 by his nephew, Captain Robert George Wardlaw-Ramsay, with a memoir by Dr W. H. Russell, and the attribution Walden is used in taxonomic listings.

He married twice: firstly in 1857 to Helene Kilmansegge (d.1871) and secondly in 1873 to Julia Mackenzie of Seaforth, daughter of Lt.-Colonel Keith William Stewart-Mackenzie (9 May 1818 – June 1881)[9][10] and of Hannah Charlotte Hope Vere.