[1] The family were however distinguished enough to bear arms, which were: Argent, a chevron between three millrinds sable, with motto "Pietate et Prudentia".
[2] Of his elder brothers, Samuel, of Queens' College, Cambridge, was a vicar of Royston, Hertfordshire, and John ‘a very considerable tradesman’ at Dedham, Essex.
Bury, where Lord-keeper North was among his schoolfellows, Isaac was admitted at St John's College, Cambridge, in 1656, and graduated M.A.
On leaving the university Milles took holy orders, and became curate in sole charge of Barley, Hertfordshire, the rector, Joseph Beaumont, master of Peterhouse, Cambridge, being non-resident.
There he made the acquaintance of Henry Dodwell, and became intimate with Martin Lluelyn, whose epitaph in Wycombe Church he wrote.
[4] Milles took pupils there, including the sons of Thomas Herbert, eighth earl of Pembroke, the new proprietor of Highclere.