[4] However, Henry Gannett in 1905 attributed the town's name to Danby, North Yorkshire, in England.

[5] Historian Willard Sterne Randall, in his biography of Ethan Allen, gives yet another name origin: that Allen named Danby "after the French Naval commander whose fleet invaded New England's waters in the French and Indian War.

[4] The village of Danby lies along the eastern border of the town, on the west side of Otter Creek, and extends east into Mount Tabor.

Dorset Peak, part of the Taconic range, lies on the town's southern border and reaches an elevation of 3,804 feet (1,159 m).

The racial makeup of the town was 98.53% White, 0.54% Native American, 0.23% Asian, and 0.70% from two or more races.