The island faced the same fate with the adjacent Tablas, Sibuyan and nearby Marinduque, having been frequented by Moro raiders from the south.

Also because of the attacks, he trained the natives to defend themselves with military armaments and ordered constructions of structures that fortified the island, earning him the moniker "El Padre Capitan".

The church's rear side was placed against the mountainside for stability while the front of the bay inlet facing the town was constructed with a continuous wall-gate for protection.

[3] On May 27, 2023, a "tragic" fire engulfed the Cathedral's L-shaped convent, parish office and rectory, "except for the old stone pillars", Bishop Narciso Villaver Abellana said.

It served as a repository-temporary museum of religious treasures, including canonical books, records and antique wooden statues of Spanish-era Christian saints.