It is situated in a former Victorian herring curing factory known as Tower Curing Works, and is now part of Maritime Heritage East, a partnership of over 30 maritime museums in the East of England.

[citation needed] The curing works was closed down in the mid-1980s, and the building lay unused for almost 20 years.

[1] Over £4.5 million was spent on refurbishing and converting the Grade II listed factory into the modern museum about the building's life as a fish factory, which opened in 2005.

[2] A town-wide exhibition of works from Peter Henry Emerson was partially on display at the Museum in 2021.

[3] A Banksy mural from Gorleston-on-Sea, which had appeared following the death of a girl in 2018 and was covered up due to "sensitivity," was moved to the museum temporarily in 2022.