The site at Millbank was originally purchased in 1799 from the Marquess of Salisbury for £12,000 by the philosopher Jeremy Bentham, acting on behalf of the Crown, for the erection of his proposed panopticon prison as Britain's new National Penitentiary.

[5] The marshy site on which the prison stood meant that the builders experienced problems of subsidence from the outset, and explains the succession of architects.

In addition to the problems of construction, the marshy site fostered disease, to which the prisoners had little immunity owing to their extremely poor diet.

In view of these problems, the decision was eventually taken to build a new "model prison" at Pentonville, which opened in 1842 and took over Millbank's role as the National Penitentiary.

[11] The three outer angles of each pentagon were distinguished by tall circular towers, described in 1862 as "Martello-like": these served in part as watchtowers, but their primary purpose was to contain staircases and water-closets.

In the Handbook of London in 1850 the prison was described as follows:[5] A mass of brickwork equal to a fortress, on the left bank of the Thames, close to Vauxhall Bridge; erected on ground bought in 1799 of the Marquis of Salisbury, and established pursuant to 52 Geo.

It was designed by Jeremy Bentham [factually incorrect], to whom the fee-simple of the ground was conveyed, and is said to have cost the enormous sum of half a million sterling.

Its ground-plan resembles a wheel, the governor's house occupying a circle in the centre, from which radiate six piles of building, terminating externally in towers.

So far as the accommodation of the prison permits, the separate system is adopted.Each cell had a single window (looking into the pentagon courtyard), and was equipped with a washing tub, a wooden stool, a hammock and bedding, and books including a Bible, a prayer-book, a hymn-book, an arithmetic-book, a work entitled Home and Common Things, and publications of the Society for Promoting Christian Knowledge.

At each quarter of an hour, day and night it chimes a bar of the "Old Hundredth" and those solemn tones strike on the ears of the lonely listeners like the voice of some monster spirit singing the funeral dirge of Time.

The principal new buildings erected were the National Gallery of British Art (now Tate Britain), which opened in 1897; the Royal Army Medical College, the buildings of which were adapted in 2005 to become the Chelsea College of Art & Design; and – using the original bricks of the prison – the Millbank Estate, a housing estate built by the London County Council (LCC) between 1897 and 1902.

Archaeological investigations in the late 1990s and early 2000s on the sites of Chelsea College of Art and Design and Tate Britain recorded significant remains of the foundations of the external pentagon walls of the prison, of parts of the inner hexagon, of two of the courtyard watchtowers, of drainage culverts, and of Smirke's concrete raft.

[18] In Henry James's realist novel The Princess Casamassima (1886) the prison is the "primal scene" of Hyacinth Robinson's life: the visit to his mother, dying in the infirmary, is described in chapter 3.

[19] Sir Arthur Conan Doyle in chapter 8 of The Sign of Four (1890) refers to Holmes and Watson crossing the Thames from the house of Mordecai Smith and landing by the Millbank Penitentiary.

Challenger means that he expects to be buried at Westminster Abbey, but his rival Professor Summerlee responds sardonically that he understands that Millbank Prison has been demolished.