[5][6] The late-medieval manor may have been constructed on the site of a previous building, based upon the presence of remnants of an earlier wall.

[2] The castle may have been built by Robert de Canteilupe III, who owned one ploughland at Candleston in 1320 according to the Despenser Survey of Glamorgan of that year.

[7][nb 2] Robert de Cantilupe III's daughter Joan, heiress of Candleston, married Sir William Horton of Tregwynt, Pembrokeshire.

[7] Sands from the dunes shifted starting in the late 14th century and encroached on nearby Kenfig Castle, resulting in its evacuation.

[1] The Royal Commission on Ancient and Historical Monuments in Wales reported that Candleston's "elevated position was just sufficient to save it from the encroaching sand dunes on the northern fringe of the Merthyr Mawr Warren, which entirely surrounded the promontory and formed a relatively stable margin close to the southern wall of the castle.

Janet Horton and Richard Cradock received Candleston, possibly because the Eyres did not have children or due to the nature of their tenure.

During his career, he was steward of Gower, Chancellor of Glamorgan and constable for life of the Kenfig and Caerphilly castles.

In the late 15th century the manor was refurbished and modified; It is believed that Mathew Cradock was responsible for the construction.

The manor was held by the Grevilles, who visited only occasionally, made only minor modifications, and sold it to the Franklen family in the 18th century.

[7] In the early 19th century modifications included construction of a stable, the castle was crenellated[5] for ornamental purposes and a defensive tower was built.