Tung Chung Fort was originally built in the Shun Hei era (淳熙, 1174–1189) of the Southern Song dynasty.

After three years of peace, the soldiers were called back and 150 of them were transferred to build Kowloon Walled City, in which they later stayed.

During the Qing dynasty, many pirates, including the well known Cheung Po Tsai, chose the bay of Tung Chung as their base and made use of the fort.

In 1832 (or 1817 alternatively) the fort was rebuilt and garrisoned by the Right Battalion of Tai Peng to defend the coast from pirates until the lease of New Territories to Britain in 1898.

There are six still intact old muzzle-loading cannons, each resting on a cement base with enclosures made of granite blocks, which measure 70 by 80 metres (230 by 260 ft).