[4] Although the present castle lacks all defensive capabilities, and was built in a period when private fortresses had become obsolete in lowland Scotland, there was possibly a tower house on the site.

Until the early seventeenth century, the Anglo-Scottish border lands, or "Marches", were a lawless place where reprisal attacks were common, and which often took the form of cattle rustling or murders, carried on by gangs of Reivers.

Floors also stands opposite the site of Roxburgh Castle, an important medieval fortress where King James II was killed during a siege in 1460.

The lands of Floors were held by the monks of Kelso Abbey, until the Reformation, when they were handed to Robert Ker of Cessford (1570–1650, later the first Earl of Roxburghe) by King James VI.

She brought with her from her Long Island home a set of Gobelins Manufactory tapestries that were incorporated into the ballroom in the 1930s, and added to the collection several modern pictures by Walter Sickert and Henri Matisse, among others.