The church ground includes a cemetery, which continues to accept burials, though there are restrictions on who can be buried there.
His brother-in-law Samuel Pole Phillips donated 12 acres of land for the church and started a fund to pay for its construction.
Construction commenced in 1850, with the first service held in the church in April 1853.
When the Phillips family returned to England worked stopped on the church and it fell into disrepair.
When they returned, the work recommenced and the church was officially opened on 19 July 1857.