Bailey was, according to Cotton Mather, who preached his funeral sermon, born near Blackburn in Lancashire on 24 February 1644.

When arraigned, he dared to address the bench thus: 'If I had been drinking, gaming and carousing at a tavern, with company, my lords, I presume that I would not have procured my being thus treated as an offender.

Must prayers to God and preaching Christ with a company of Christians who are peaceable, inoffensive, and serviceable to his majesty and the government, as any of his subjects — must this be considered a greater crime?'

At length intimation was secretly sent him that he would be allowed out on condition that within a limited specified time he left the country.

Early in 1685 he was in correspondence with the Independent congregation at Watertown, Connecticut, with the result that on 6 October 1686 he succeeded the Rev.

Judge Samuel Sewall's Diary records that Bailey, holding to the validity of his original ordination, refused to be inducted with the laying on of hands — an innovation in Independent church ways that was something of a scandal for the moment.

He died on Sunday, 12 December 1697, and Cotton Mather preached his funeral sermon, which was published.

He chose for its text the words 'Into Thy hands I commit my spirit,' on which Mr. Bailey had prepared a sermon — never delivered — under a presentiment that it would be his last.

A short book by him was published by friends, 'Man's Chief End to Glorify God, or Some Brief Sermon-notes on 1 Corinthians x.

A portrait of him (in oils) was in the possession of the Massachusetts Historical Society, in whose archives were also some manuscripts of his brother Thomas.