Due to disagreements with Massachusetts' state Congregationalist (Puritan) church, Gould absented himself from Sabbath meetings.
In 1655 he refused to allow his child to be baptized by the Congregationalist church in Charlestown, Massachusetts, where he was living and starting meeting with Henry Dunster, another anti-pedobaptist who was the first president of Harvard College.
Another disagreement took place in the Cambridge church, and anti-pedobaptists began meeting in Gould's home.
In 1665 Gould drew up a statement of faith for a Baptist congregation, which began meeting in secret on Noddle's Island (in Boston Harbor).
The congregation was mainly Calvinist in theology and borrowed heavily from earlier Baptist confessions of faith.