Richards was born in 1790[1] in Dublin, the younger son of John Nunn Richards (died 1821), solicitor, and his first wife Elizabeth Fitzgerald, daughter of Oliver Fitzgerald.
The Richards family of Macmine Castle, Wexford, were close relatives.
Richards married firstly in 1815 Catherine Moloney, daughter of Henry Gonne Moloney, barrister, of Tulla, County Clare, and his wife Caroline Walker, by whom he had seven children, three sons and four daughters:[4] John, William, Oswald, Elizabeth who married a Mr. Rolleston, Catherine who married Thomas Spunner, Charlotte who married a Mr. Symes, and Henrietta-Caroline, who married Robert King Piers, nephew of Sir Robert King, 2nd Baronet of the King Baronets, of Charlestown, in 1844.
One of the younger John's sons, Sir Henry George Richards, also became a judge, serving as Chief Justice of the Allahabad High Court, India, from 1911 to 1919.
In Dublin he lived in Dalkey, and later Dundrum, and he also had a country house in County Clare.