He sometimes assisted William Haworth, then minister of a congregation of dissenters in Hertford, and succeeded him in the charge 27 September 1705.
He was an active member of the King's Head Society, which was formed for the purpose of assisting young men to obtain academical training for the ministry.
Two Coward lectures, which he published in 1729 under the title of 'Christ the Son of God,' were attacked by Samuel Chandler in 'A Letter to the Rev.
Guyse replied with 'The Scripture Notion of preaching Christ further cleared and vindicated in a letter to the Rev.
Besides the works mentioned above he wrote the following: John Wesley stated in the Preface to his Notes on the New Testament that he was indebted to Dr. Guyse for many 'useful observations'.