In medieval times, it communicated through flag or mirror messages with the nearby Bonot Tower and the Villa Castle in Challand-Saint-Victor.
The fief of Graines is documented since 515, when King Sigismund of Burgundy donated it to the newly formed Abbey of St. Maurice.
In 1263 the abbey sold the fief to Godefroi of Challant, a faithful vassal of the House of Savoy, whose family held the castle until the 18th century.
When the Challant disappeared in the 19th century, the castle was bought by Passerin d'Entrèves family, who later sold it to the commune of Brusson.
It had an irregular line of walls measuring c. 80 x 50 m, which housed a series of structures such as the large donjon (square tower) and a small chapel, the only ones remaining now.