He was the son of Joseph Gulston, a London merchant trading in Lisbon.
Together with his fellow MP Thomas Calcraft, he is credited with financing Poole Guildhall.
She was the daughter of a Portuguese merchant named Sylva and had come to London with the family on their return.
[2] The marriage was not acknowledged for many years, principally owing to the elder Joseph Gulston's dread of his sister, and for some time his children, two sons and two daughters, were brought up in the strictest concealment.
His son Joseph followed him as MP for Poole and inherited his estates, including Ealing Grove in Middlesex, which he would later rebuild.