According to the account by the Continuator of Wace and others,[3] in his youth Brictric declined the romantic advances of Matilda of Flanders (c. 1031 – 1083), later wife of King William the Conqueror, and his great fiefdom was thereupon seized by her.
[4] The principal sources of the barony's lands in Devonshire were from the former holdings of:[5] Matilda of Flanders (c. 1031 – 1083), later wife of King William the Conqueror, seized Brictric's lands which after her death in 1083 reverted to the royal demesne of her eldest son King William Rufus (1087–1100).
The feudal barony proper was created when William Rufus granted the lands to his follower Robert FitzHamon (died 1107),[7] the conqueror of Glamorgan.
Maud (or Mabel) FitzHamon the daughter and sole heiress of Robert FitzHamon brought the barony to her husband Robert de Caen, 1st Earl of Gloucester (pre-1100-1147), a natural son of King Henry I (1100–1135), the younger brother and successor of King William Rufus.
Following the death of William Fitz Robert, 2nd Earl of Gloucester, in 1183, the barony escheated temporarily to the crown and was controlled by King Henry II (1154-1189) until 1189.