David James was educated first at the Leicester Collegiate School, under William Hepworth Thompson, afterwards Master of Trinity College, Cambridge, and in August 1840 he went to Rugby, first under Arnold and then under Tait.

Vaughan, Davies, and Brooke Foss Westcott, all fellows of Trinity, formed at Cambridge a lifelong friendship.

In 1853 Vaughan was ordained deacon, and began his pastoral work in Leicester, living on his fellowship, and serving as honorary curate, first to his eldest brother at St. Martin's, and then at St. John's church.

The living was then in the gift of the crown, and had been held by his father and two of his brothers continuously since 1802, save for a short interval of twelve years.

His efforts to elevate the working classes by means of education were no less earnest and successful than those of Maurice and his colleagues in London.

In 1868 there were four hundred adults under instruction, and the name of the institute was changed to 'college' as being in Vaughan's words 'not only a school of sound learning, but also a home for Christian intercourse and brotherly love.'

On Sunday afternoons, Vaughan gave in St. Martin's church addresses on social and industrial as well as religious themes to working men, including members of the great friendly societies in Leicester, and students of the college.

Vaughan was chairman of the first Leicester school board in 1871, and exercised a moderating influence over stormy deliberations.

He continued to act as chairman of the Institution of District Nurses, president of the Working Men's College, and honorary chaplain to the isolation hospital.

He died at the master's house at Wyggeston's Hospital on 30 July 1905, and was buried at the Welford Road cemetery, Leicester.

He married, on 11 January 1859, Margaret, daughter of John Greg of Escowbeck, Lancaster; she died on 21 February 1911 and was buried beside her husband.

After his death, a new Vaughan Working Men's College, situate in Great Central Street and Holy Bones, Leicester, was erected as a memorial to him at the cost of 8000l.