Gelston was born on July 4, 1744, in Bridgehampton in Suffolk County on Long Island in what was then the Province of New York.

He signed the Articles of Association in 1774, agreeing to avoid British imports, even though this hurt his own business.

As speaker, he took a leading role in reconciling the differences between Tory and Whig factions.

He oversaw the repeal of all the laws that had imposed civil and legal penalties on Tories.

[4] In 1801, Gelston was appointed Collector of the Port of New York by President Thomas Jefferson, and held that post until 1821 when he retired.