[3] In 1862 Charles Castleman moved to Glasshayes with his third wife Isabel Swinburne,[6] and whilst in residence gifted the clock to the clocktower of the newly built local St. Michael and All Angels church, in exchange for the closing of an insalubrious public road which ran directly behind the house.

Supposedly, every year on the night of his death (July 7) strange music can be heard in certain rooms, and folklore has it that this is from a grand ball that the Duc holds annually for the dead.

[3] From its earliest incarnation Glasshayes has been notable for its eccentric "octagonal tower," built as an original feature by George Buck to serve as a lookout for local smuggling operations (common within the New Forest area).

When the Duc de Stacpoole purchased the property he continued to run the area smuggling ring from the house, also using his yacht "the Gipsey Queen" to transport illicit goods.

A raid was carried out by excise agents in 1847, during which the Duc barricaded himself into the octagonal tower with a military saber, and the siege was widely reported in national papers of the time.

The house continued to be used for smuggling activities during its years of non-residence after the Duc's death, but seems to have ceased the association from Charles Castleman's tenure beginning 1862.

Local tradition reports numerous smuggling caches and hollows throughout the original portion of the property (particularly in the floor and behind the wooden paneling of the surviving octagonal room).

[3] In 2014 the hotel and estate was closed and bought by developers PegasusLife, who plan to demolish the house in order to build retirement flats on the land, despite local opposition.

[20] In view of the recently discovered connection with novelist Sir Arthur Conan Doyle, who designed the garden facade, a new application was made in 2017 to list the hotel.