As early as 1780, black members of the John Street Church were holding separate class and prayer meetings.
The group met for prayer on Sunday afternoons and heard preachers and exhorters on Wednesday evenings in a house in Cross Street, which they remodeled to hold these meetings.
This incorporation placed the church and its property firmly under the control of the trustees, who were required to be of African descent.
It acquired a burial ground in 1807 and laid plans to buy the lots it had leased, along with another adjacent one, and to erect a new brick church to replace the original building.
In 1820, as Zion was engaged in erecting its new church, the congregation was scattered across a number of temporary meeting places, a competing black denomination appeared in Richard Allen's African Methodist Episcopal Church, that was trying to build a national organization from its Philadelphia base.
On September 13, 1820, Abraham Thompson and James Varick were selected by the congregation to become elders and they began to act immediately, holding communion services.
Finally, on June 17, 1822, white Methodist elders ordained Abraham Thompson, James Varick, and Leven Smith.
On July 4, 1827, the thanksgiving service for the final abolition of slavery in New York was held in Zion church.
His remains now repose in the crypt of the Mother African Methodist Episcopal Zion Church in Harlem.