In later life his judicial bearing was marked by an eccentricity of manner, but although his decisions were frequently the subject of comment, very few of his judgments were reversed on appeal.
As a member of the Law Amendment Society he was a constant attendant at the weekly meetings in Lancaster Place.
Fane was an ardent lover of field sports, and was well known in the Leicester hunts; he was also a patron of the fine arts, and possessed a collection of paintings.
; she died at Rolls Park, Chigwell, on 15 December 1838; and secondly, on 7 September 1841, Harriet Anne, only daughter of Admiral the Hon.
By his second wife he had two daughters, Cicely Harriette Fane (1846 – 1933) and Blanche Anna Fane (1847 – 1935), and two sons, Lionel Arthur (1851 - 1872) and Cecil Francis William (1856-1914) who married 8 May 1880 Lady Augusta Fanny Rous (1858 – 1950), daughter of John Rous, 2nd Earl of Stradbroke.