The Old Red Mill was then bought for $200 in 1840 by Joseph Congdon of Shelter Island, and it was then transported across Peconic Bay in pieces by an ox team and barge.

Fiske moved the windmill from its original location near the center of town to his estate, Sylvester Manor, in Shelter Island, where it remains to this day.

The mill was built in 1810 at Southold for a company whose partners were Benjamin Horton, Nathaniel Overton, Moses Cleveland, Barnabas Case and Joseph Halliock.

[5] The windmill was purchased in 1840 by Joseph Congdon and moved to Shelter Island where it stood at the center of the village, near the library and high school.

The mill was put back in operation during 1917–1918 to provide meal and flour for the inhabitants of the Island during the food conservation period of the First World War.