The Grade I listed building is located within Christchurch Park and sits by the southern gates close to the town centre of Ipswich.

Christchurch Park was originally the grounds of the Priory of the Holy Trinity, with an area of many square miles, coming up to the medieval town walls.

A survival from the original is the house's Latin motto, on a stone dated 1549:"Frugalitatem sic servas, ut dissipationem non incurras.

Woolnough, an active member of the Museums Association and of the Savage Club, also developed its use as a venue for annual meetings of various Societies, and for educational purposes.

The Thomas Gainsborough Bicentenary Exhibition of 1927, including also works by George Frost and John Constable, showcased the mansion as the home of a collection of national importance.

The structure was further enlarged with the addition of the new Wolsey Art Gallery at the rear, and an entire two-storey wing rescued in 1924 from the demolition of a Tudor merchant's house.