According to National Park Service architectural historian Carolyn Pitts, "Cape May has one of the largest collections of late 19th century frame buildings left in the United States... that give it a homogeneous architectural character, a kind of textbook of vernacular American building.

He landed on the shore of Delaware Bay a few miles north of Cape May Point before returning to the Atlantic Ocean.

[8] The early emergence of Cape May as a summer resort was due to easy transport by water from Philadelphia to the Atlantic Ocean.

Early Cape May vacationers were carried to the town on sloops from Philadelphia, and water transport was also easy from New York, Baltimore, Washington, D.C. and points south.

In 1830 a visitor wrote that Cape May Island is a noted and much frequented watering place, the season at which commences about the first of July and continues until the middle of August or the first of September.

Serving Presidents who visited included Franklin Pierce (1855), James Buchanan (1858), Ulysses Grant (1873), Chester Arthur (1883), and Benjamin Harrison (1889).

Newport, Rhode Island, Saratoga Springs, New York and Long Branch, New Jersey were the town's main rivals in the summer resort business, as Cape May's reputation rose and fell with the whims of fashion.

[9] Architect Stephen Decatur Button began designing buildings in Cape May in 1863 when he remodeled and expanded the Columbia Hotel.

Architect Frank Furness is believed to have designed the Emlen Physick Estate, but may have otherwise visited Cape May only as a vacationer.

Having lost its transportation advantage with the coming of the railroad and the automobile, Cape May fell out of fashion as a popular resort.

Atlantic City became the popular New Jersey beach resort in the 1920s and in the 1950s and 1960s the automobile-oriented Wildwoods, just north of Cape May, became a strong competitor, with its own distinctive architecture.