In Samuel Tymms's Handbook of Bury St. Edmunds it is stated that Daniel Defoe was an attendant on his ministry.
[1] In 1696, Bury was engaged in collecting a list of the nonconforming ministers; Oliver Heywood supplied him (14 August) with the names in Yorkshire and Lancashire, through Samuel Angier.
On 11 August 1700, John Fairfax, ejected from Barking-cum-Needham, Suffolk, died (aged seventy-six) at his house in that parish; Bury preached two funeral sermons for him; the one at the actual funeral at Barking was, by an unusual concession, delivered in the parish church.
[1] In the Hewley suit, 1830–42, efforts were made by the Unitarian defendants to collect indications of concession to heterodox opinion on the part of Bury, as a representative Presbyterian of his time.
Thomas Smith James's History of the Litigation and Legislation respecting Presbyterian chapels claimed that the 'Exhortation' at Savage's ordination, quoted to prove opposition to the Calvinistic Doctrine of Election, was not by Bury, but by John Rastrick, M.A., of Lynn (died 18 August 1727).
In a farewell letter from Bury to his Lewin's Mead congregation, he says, 'I never was prostituted to any party, but have endeavoured to serve God as a catholic Christian,’ and speaks of requirements which have no good Scripture warrant, as making 'apocryphal sins and duties.'