Another violent hurricane struck in 1780 and according to Robert Hermann Schomburgk's History of Barbados only two churches and one chapel remained; St. Andrew, St. Peter and All Saints.

Schomburgk states that in 1846 at the time of his writing this book, the building was 54' X 40' in size with a seating capacity of 550 and a congregation of approximately '360 souls'.

After nearly 200 years the walls of the original stone building began to decay and were partly demolished and replaced by a larger, more substantial structure in 1874.

The church's first organ was ordered and built by the English firm, Hill & Son, at a cost of 320 pounds.

Over time this organ was replaced, with various upgrades performed over the years until the existing instrument was completely rebuilt from 2006–2007 and rededicated in 2008.

However, considering its historic value the old one was sent to England for repairs and returned to Barbados to be retained as a monument, which predates the American "Liberty Bell" by 56 years.

The Baptistery, on the ground floor of the belfry, contains the original font which is over 300 years old and bears the inscription "Debit Richardus Walter to the Church of St. James Anno 1684."