According to a February 2019 report: "The castle was probably built between 1566 and 1572 by George Sinclair, 4th Earl of Caithness [and] includes a dominating tower with a series of tall ranges to the side and rear creating a three-sided courtyard open to the north and the sea.

[3] The castle passed to Lord Caithness's younger son William, founder of the Sinclairs of Mey, although it later became the seat of the earls.

[5] The castle's name was changed to Barrogill, and the structure was extended several times, in the 17th and 18th centuries, and again in 1821 when Tudor Gothic style alterations were made, to designs by William Burn.

The Trust would manage the property; its mandate was "to secure the future of the building, advance historical and architectural education, to develop the native breeds of Aberdeen Angus and Cheviot sheep and to undertake projects for the benefit of the local community", according to one report.

Maintenance work completed by the Trust in 2018, including roof repairs, rewiring of the interior and lime harling of the exterior.

[14] The Foundation stated its intention to retain the Trust's goals: "...the preservation of buildings and monuments; the promotion of historical and architectural education; the preservation of the Aberdeen Angus breed of cattle, and the championing of wider benefits to the community, while the Duke of Rothesay and The Prince's Foundation is certain to make the continuation of his grandmother’s legacy a priority.

"[15]In early May 2019, the Prince of Wales (now Charles III) formally opened a new building, Granary Lodge,[16] as a 10-bedroom bed and breakfast on the castle grounds.

[19] In January 2023, it was reported that the castle tearoom would be open to the public every Tuesday until March as a warming centre, an initiative by the Prince's Foundation.

The Queen Mother's purchase of the castle is featured in the Netflix show The Crown (episode 8 of the first season, "Pride & Joy").