One of Hart's early publications was a tract denouncing Christianity (prior to his conversion) called The Unreasonableness of Religion, Being Remarks and Animadversions on the Rev.
He preached at Jewin Street chapel in London, a building with multiple galleries, to a congregation of significant size.
Hart wrote of his early life in his autobiography: As I had the happiness of being born of believing parents, I imbibed the sound doctrine of the Gospel from my infancy; nor was I without touches of the heart, checks of the conscience, and meltings of affections, by the secret strivings of God's Spirit within me while very young; but the impressions were not deep, nor the influences lasting, being frequently defaced and quenched by the vanities and vices of childhood and youth.
It was then that he wrote The Unreasonableness of Religion, in an effort to convince John Wesley that he should not be doing good works only believing in God.
But soon after this, Hart again began to be afraid of the life to come (eternity), and feared exceedingly when reading about the condemned in passages in the Bible.
Hart's motto after this time was: "Pharasaic zeal and Antinomian security are the two engines of Satan, with which he grinds the church in all ages, as betwixt [between] the upper and the nether [lower] millstone.
[3] The inscription on his monument reads: Joseph Hart was by the free and sovereign grace and Spirit of God raised up from the depths of sin, and delivered from the bonds of mere profession and self-righteousness, and led to rest entirely for salvation in the finished atonement and perfect obedience of Christ.