It was later restored unto his son, Edward Courtenay, the last earl of this family, by Queen Mary I of England, and upon his death, was left to descend to the heirs general, who sold this manor and house.
Now the manor was divided; the one moiety the lord Petre possesseth, the other moiety, Sir John Pole and Sir John Drake, knights, acquired.Hoskins states that it was first built during the reign of King Edward I of England (1272–1307) by Hugh de Courtenay (1276–1340),[1][2] and was rebuilt ("on a magnificent scale", according to Hoskins[1]) by Henry Courtenay, 1st Marquess of Exeter, 2nd Earl of Devon (1498–1539), after whose attainder and execution, it became forfeit to the crown.
It was restored with other lands to his son, Edward Courtenay, 1st Earl of Devon (1527–1556), after whose death it passed, together with Tiverton Castle, to his heirs, the descendants of his four great-aunts, and eventually fell into a ruinous state.
His son Sir William Pole (1561–1635), the antiquary, purchased the remaining shares from the heirs of the Courtenays, and rebuilt the house, as he recorded after 1618 in his "Collections towards a History of Devon": A goodly building was here intended by the last erles but altogether unfinished, and nowe the whole being reduced from all ye co-heires unto my possession, I have newe built the howse and made it the place of my residinge".
[5] During the Civil War it served in 1644 as the headquarters of Prince Maurice (d. 1652), who used it as a base for his attack on Stedcombe, then being used as a garrison for parliamentary troops by its owner Sir Walter Erle.
[4] During the conflagration, many of the volumes of the manuscripts of the antiquary's great work on the history of Devon, with substantial additions made by his son the 1st baronet, were destroyed.
However he died before his plan could be effected and stipulated in his will that his heirs should build "a good substantial dwelling-house with suitable offices and at least five acres of land in Colcombe Park where his widow should live".
Only one small building, now converted to a house with Grade II listing, displays signs of ancient construction, namely its mullioned stone window.
Perhaps little if any of the edifice wherein once the Courtenay Earls of Devon resided is now to be seen...Of this house of Colcombe, which Sir William (Pole) called a Castle are the present ruins.
The whole of the area seen in the preceding sketch (i.e. dated 26 January 1795) and consisting of the interior part of the building are nearly the same as the print given in the "History of Devon" (i.e. by Polwhele).
On the western side, separated by a lane, is the house in which the farmer dwells who rents the estate of Colcombe of Sir J. de la Pole and which was probably built from the ruins of the castle.