Upon the hill is St. Catherine's Oratory (known locally as "the pepperpot"), which is a stone lighthouse built in the 14th century by Walter De Godeton.

Reportedly, de Godeton was found guilty for having plundered wine that belonged to the Church from the shipwreck of the St. Marie of Bayonne in Chale Bay.

He was ordered to make amends, under threat of excommunication, by building and maintaining the lighthouse.

It was completed after his death, and staffed by a priest; fires were lit in the tower to warn ships of the coast.

After Hoy's death, an inscription was made at the base that commemorates soldiers killed in the Crimean War in 1857.