The gallery hosts one of the most significant public art collections in the South of England and draws over 100,000 visitors a year.

A noted historic building in Eastbourne, the Manor House is dwarfed in age by its neighbours; being opposite the Lamb Inn, dating back to 1180AD[6] and St Mary's Church, from the same period.

[7] Despite having been owned by the town as a public facility for most of the 20th century, the Manor House and gardens were sold in 2005/6 by the local authority, Eastbourne Borough Council.

Designed by Rick Mather Architects and built mainly from concrete, at a cost of £8.6m,[9] the new gallery building was planned to be more easily accessible to the public and to store the 4,000 works of the growing collection in a safe and climate-controlled manner.

Unlike the old building, no provision was made to incorporate a local museum, though substantially more space was created for a café, shop and larger exhibitions.

It boasts in excess of 5,000 works of art by historic, modern and contemporary artists including: Lawrence Alma-Tadema, John Gascoigne Lake, Vanessa Bell, David Bomberg, Alan Davie, Tacita Dean, Olafur Eliasson, Anya Gallaccio, Thomas Jones, Peter Liversidge, Harold Mockford, Henry Moore, Cedric Morris, William Nicholson, Julian Opie, Ian Potts, Victor Pasmore, Pablo Picasso, Eric Ravilious, Eric Slater, Wolfgang Tillmans, Alfred Wallis, Christopher Wood, Joseph Wright of Derby and Carol Wyatt.

The current collection now includes oil paintings, watercolours, works on paper, etchings, prints, sculpture, wood cuts, ceramic objects, installations and video art.

Since the gallery was established it has received donations of work by significant artists including: Walter Sickert, Pablo Picasso, Henry Moore, Victor Pasmore, Alfred Wallis, Frances Hodgkins, Phelan Gibb and David Bomberg.

The Towner holds the broadest collection of paintings, illustrations and commercial designs in the world, by this important modern British artist of the early 20th century.