He was born on August 14, 1745, in Kingston, Jamaica, the only child of Michael Hay from Edinburgh, Scotland and Esther Wilkins.

[1] He was named after his great aunt, Ann Mister, sister of Martin Wilkes, father of his mother.

[2] Hay was a colonel in the 2nd regiment of the Orange County, New York militia and observed enemy operations in the Haverstraw Bay of the Hudson River.

[5] Later, in the fall of 1777, his house and farm buildings in Haverstraw, New York, were burned down by British raiders.

[1] After several unsuccessful attempts to start a family, the couple returned to Haverstraw and purchased approximately 200 acres of land near the Minisceongo Creek in December 1773.