It is not possible to say exactly when the first Scott's Hall at this location was built but there is certainly evidence to suggest that it replaced an old medieval house and was here in 1429 when the Scot family moved into it from the nearby village of Braborne (sic).
Diplomatically, William later dropped the de Balliol part of his name in order to avoid the anger of King Edward of England, retaining only that of Scot.
One of his grandsons, John le Scot, married Caroline Carter and thereafter moved to Kent and eventually Scott's Hall.
For nearly four hundred years the Scot family held considerable influence over Kent and had positions at court according to their rank.
John le Scot was a committed supporter of the House of York and, along with many of his descendants, is buried at St Mary's Church in Brabourne where they had originally briefly settled when they first came to Kent.
Little evidence remains of how well the Scot family got on together, except amongst the Deeds four letters were found dated 26 February, 13 March, 16 April, and July 1779.
His daughter Joan Scott married Thomas Yerde of Denton Court and the manor of Tappington in Kent and East Cheam in Surrey.
He married Agnes Beaufitz (d.1486/7), daughter and co-heiress of William Beaufitz (alias Bewfice) of The Grange, Gillingham, Kent, and likely also a fishmonger of London, by whom he had a son and two daughters including: Sir William Scott (1459-1524) of Scot's Hall in Smeeth, Kent was Lord Warden of the Cinque Ports.
He served in King Henry VIII's campaigns in France, and was active in local government in Kent and Burgess of New Romney.
He was the grandfather of both Reginald Scot, author of The Discoverie of Witchcraft, a source for Shakespeare's Macbeth, and Thomas Keyes, who married Lady Mary Grey.
He married three times: Sir John Scott (1570-1616), of Scot's Hall and of Nettlestead Place in Kent, was an English soldier, Member of Parliament (MP) and an early investor in the Colony of Virginia.