Florence, the youngest and favorite grandchild of the transportation tycoon Cornelius Vanderbilt, married Twombly in 1877 after meeting him at the two families' summering spots in Newport, Rhode Island.

[2] In 1893, the couple commissioned the famous firm of McKim, Mead & White (the architects of the Old Penn Station and the Rhode Island State House) to build Florham as the country setting to raise their family.

"[2] Beginning in 1891, some 1,200 acres (490 ha)[3] were acquired in 37 purchases to assemble a property on Madison's "millionaire's row", a neighborhood that was home to several other Gilded Age estates belonging to the Rockefellers, Dodges and Mellons.

[2] Florham's design was principally inspired by Sir Christopher Wren's late 17th-century expansion of Hampton Court Palace under King William III and Queen Mary II, evident especially in the house's lay-out, pillars, and contrasting stone and red brick.

The furnishing of the house's interior, overseen mainly by Stanford White, included several Barberini tapestries, a Louis XV ballroom and several Renaissance fireplaces purchased from noble Italian homes.

Setting the site of the mansion on a hill overlooking the Passaic Valley and Black Brook meadows, Olmsted told Twombly "You have a sweep of landscape to an infinitely remote and perspectively obscure background ... as much so as if you owned the state of New Jersey.

Many of these items remain in the White House collection, including several chairs from the estate, which have sat in the Oval Office under Presidents George W. Bush, Barack Obama and Donald Trump.

[10] Florence and her daughter Ruth died in 1952 and 1954, respectively, and the estate was broken up, with the bulk of the lower portion, occupied by the dairy farm and stables, going to Exxon, later to be sub-developed into a corporate park.