[2] Renovated by Margaret Ogilvie in 1601, the tower passed by marriage into the Oliphant family in 1707, but was neglected and gradually fell into disrepair after 1715.

[3][4] Only the northern part of the castle now remains, in the form of a two-storied tower with a garret above a vaulted ground floor.

The fireplace of what would have been the great hall is exposed on the south face of the surviving structure at the level of the first floor,[1][4] alongside a substantial aumbry.

The walls, which are intact to the level of the roof, are made of rubble with ashlar detailing, and up to 0.9 metres (3.0 ft) thick in places.

[5] Milton Tower was the birthplace of St John Ogilvie, Scotland's only post-Reformation Catholic saint.