Atterbury studied the harpsichord, composition, and harmony in the leisure time he could spare from his business, which was carried on in Turn Again Lane, Fleet Market.
He acquired considerable proficiency in music, and on the death of his father, being left tolerably well off, gave up his business and retired to Teddington.
On 5 May 1773, he produced at the Haymarket theatre an oratorio, 'Goliah' which failed disastrously, though it was afterwards repeated at West Wycombe on 13 Aug. 1775, on the occasion of the burial of the heart of Paul Whitehead in the mausoleum of Lord Le Despencer.
[1] In 1784 Atterbury sang in the chorus of the Handel commemoration, and in 1787, on the establishment of the glee club at the Newcastle Coffee House, Castle Street, Strand, his name occurs as one of the original members.
He was at this time still living at Teddington, but his improvidence forced him to remove to Marsham Street, Westminster, and to give concerts in aid of his finances.