[1][2] After the area became a municipal borough in 1890, civic leaders decided this arrangement was inadequate for their needs and that they would procure a purpose-built town hall: the site chosen for the building was occupied by the Castle Hotel in Richmond, which was purchased by Sir John Whittaker Ellis, the local member of Parliament, and was donated by him to the Richmond Vestry in 1888.

[4] The design involved an asymmetrical main frontage with six bays facing onto Whittaker Street; the central section featured an arched entrance flanked by Doric order pilasters on the ground floor; there was an oriel window above the doorway with niches on either side; there were three smaller windows above the entrance on the second floor.

A turret was installed at the north west corner of the building at roof level and a projecting clock, made by the Leeds firm of William Potts & Sons, was installed on the Hill Street frontage;[4] a set of bells were provided by John Taylor & Co of Loughborough on which the clock chimed the quarters and struck the hours (but the council voted to discontinue use of the bells in 1952).

[3] A bust of Sir John Whittaker Ellis by Francis Williamson was unveiled by Mary Adelaide, Duchess of Teck on the staircase of the building in 1895.

[3] Although some of the building's ground floor was subsequently leased for retail use, most of the former town hall was retained to accommodate the Central Reference Library and the Museum of Richmond, which was formally opened by Queen Elizabeth II on 28 October 1988.