It was built by Sir Edward Carey, the keeper of the Jewels to Queen Elizabeth I from stones removed from Berkhamsted Castle.

Several notable residents of Berkhamsted lived in the house and over the years its owners welcomed guests such as King Charles I and William Gladstone.

[2] Berkhamsted Castle had, by this stage, fallen into disrepair and the lease of a ruined castle was possibly intended as some sort of royal joke; Carey owed a nominal annual rent of one red rose, payable yearly on St John the Baptist's Day (24 June).

The survey noted that a commemorative stone could be seen below the drawing-room windows on, also facing south-east, bearing the inscription "1611", possibly marking the alterations made for Prince Henry when he purchased the house.

It is known that the young Prince Charles, then aged sixteen, paid a visit to the Murrays on 14 August 1616, when they spent an afternoon hunting in the estate, Berkhamsted Park.

[7][8] The turbulent events of the English Civil War came to Berkhamsted Place in the 1640s when the Murrays' daughter, Ann Murray, became in a Royalist plot to protect the life of the King's second surviving son, the young Duke of York (who was later to become King James II of England, from the Parliamentarian forces.

Possession of the house was taken by a Berkhamsted-born soldier in Cromwell's Army, Lieutenant Colonel Daniel Axtell, aged 26, had risen in the ranks and served as Captain of the Parliamentary Guard during the execution of King Charles I.

Ann Murray was later honoured for her loyalty to the crown when King James II granted her a royal pension.

Reconstruction was hastily commenced, probably financed by John Sayer, a wealthy local man who was Chief Cook to King Charles II.

The two ladies entertained many notable establishment figures in their fashionable country residence, including the Duke of York (who was later crowned King George V) and the Prime Minister William Gladstone.

The surviving 17th-Century wing of the house became the studio of the renowned sculptor Reg Butler, whose 1953 work Unknown Political Prisoner won the Grand Prize in a competition held by the London Institute of Contemporary Arts.