According to his wishes, Miller was buried on his estate in a mausoleum built after his death, and decorated with sculptured friezes by Alfred Gatley, subsequently referred to as the Craigentinny Marbles.

As a book collector, Miller was regarded as the successor of Richard Heber, and many of the rarest works from his collections of the latter passed into the library which he formed at Britwell Court, near Burnham, Buckinghamshire.

He was particular in his choice of copies, and from his habit of carrying about with him a foot rule to measure the size of a 'tall' copy of a book which he wished to buy, he became known at sales and among collectors as 'Measure Miller.’ The Britwell Library, formed chiefly at the time of the dispersal of the Heber and other important collections, and then added to by acquisitions from Thomas Corser, Laing, and other sales, was unrivalled among private libraries for the number, rarity, and condition of its examples of early English and Scottish literature.

It contained six works from William Caxton's press, many printed by Wynkyn de Worde and Richard Pynson, and the greater part of the Heber collection of ballads and broadsides.

It was especially rich in early English poetry, and possesses also the finest and most complete series in existence of Theodor de Bry's collections of voyages to the East and West Indies.