Collections include local and social history, the history of the castle, and a selection of local art.

The museum is housed in the former governor's house, a Grade II listed building, said to date from 1779[2] inside Haverfordwest Castle and facing the former town prison.

In 2012 Pembrokeshire County Council decided to sell the current site for use commercially, and relocate the museum to another site.

[3] The site has not been sold and the museum is still open.

In November 2023 the roof of the museum was damaged by lead thieves.