Its history is obscure: originally it was an oblong building of two storeys, with two turrets later added to the north corners and a south wing constructed.
[2] The castle has been identified with the 'house in the form of a tower' that either David de Lindsay, justiciar of Lothian, or his young namesake, the ward of Alexander of Scotland, was building in Tynedale in 1237, to the alarm of the sheriff Hugh of Bolbec; however, for this there is no positive evidence, and as David had been granted Chirdon (commanding a more important crossing of the Chirdon burn than Dally does) and one would have expected him to build there.
Later in the thirteenth century, the building (which may have been left unfinished) was completed with the addition of a north-west corner turret and south wing, and its defence was moved from the ground floor to crenellated parapets at roof level; at that point the loopholes no longer needed to be carefully built up, except one which was enlarged to light a chamber with a fireplace.
The building continued in habitation till the sixteenth century or later, as is proved by the helmet, sword point, and 'fairy' tobacco pipes found under the debris of the north-west turret in 1888; however, it had become a ruin before Camden visited it in the reign of James I.
The Armstrongs' map shows it as a ruin, and apparently during the eighteenth century the roofless walls collapsed and all visible and easily accessible stones were removed to help build Dally Mill.
[citation needed] In 1888, Mr. W. L. Charleton removed enough of the debris to reveal the remains hereafter described, but most unfortunately no proper record was kept of his funds and the owner allowed a set of 'fine columns', probably the supports of the hall roof, to be taken away to build a piggery.
The upper and lower halves of the valley watered by the Chirdon are separated by a natural barrier in the shape of a high steep-sided ridge or "kaim" pierced at east end by the stream.
The walls of the main block are from 5+3⁄4 to 6 feet (1.75 to 1.83 m) thick and contain 2-inch (5.1 cm) fish-tailed loopholes as shown on the plan and detail drawings.
At the south corner, a narrow door with chamfered jambs and sill gives entrance to what must have been either a mural stair or a small turret; this formed a weak point both structurally and strategically and was twice strengthened and reconstructed.
[citation needed] The north and south sides of the building have double base courses externally, but these are not returned across the gables.
Somewhere also at the west end was a neat chimney head with four small outlets for smoke, and if this was really David de Lindsay's stronghold we know from Hugh of Bolbec's description that it had parapet walks round its roof.
[3] Dally Castle was purchased from the Earl of Suffolk in 1664 by William Charlton of the Bower, and in 1841 when the rest of the estate passed to Sir Edward Haggerston.