Captain Francis Light, R.N., took possession of Penang Island from the Sultan of Kedah in 1786 and built the original fort.

It was a nibong (a Malay term meaning 'palm trunk') stockade with no permanent structures, covering an area of 417.6 square feet (38.80 m2).

Even though the fort was originally built for the British military, its function, historically, was more administrative than defensive.

The first recorded marriage here took place that same year when John Timmers married Martina Rozells, Light's widow.

The largest, known as Seri Rambai, was cast in 1603; in 1606 the Dutch East India Company gave it to the Sultan of Johore.

The British seized Seri Rambai in 1871 as booty after a punitive raid on Kuala Selangor, and took the cannon to Penang.