Until 1814, the Custom House stood at Sugar Quay in the parish of All Hallows Barking, immediately to the east of the present site.

Around 1380, one John Churchman built a custom house there to collect dues for the City of London, and in 1382 the Crown came to an agreement to use its facilities.

The new building was short-lived: in January 1715 a fire, which began in a nearby house, damaged it beyond repair, and a new, larger structure was built to the designs of Thomas Ripley, "Master-Carpenter" to the board of Customs.

[1] With the growth of trade, the opening of the docks, and the increases in duties during the Napoleonic wars, larger premises became necessary in the early nineteenth century.

The attic storey of the latter was decorated with terracotta figures in bas-relief by John Charles Felix Rossi and J. G. Bubb[4] representing the arts and sciences, commerce and industry, and inhabitants of various countries of the world.

[6] In 1825, the timber pilings which served as foundations for the custom house gave way, leading to a partial collapse of the building.

On investigation, it soon became clear that the building contractors Miles and Peto had grossly underestimated the cost of the work and had started to cut corners.

[7] Further investigation showed:[7] The poor quality of some of the workmanship prompted questions in Parliament in 1825 with the Chancellor of the Exchequer declaring the most scandalous frauds had been practised.

[12] In 2023, the long lease on the building was purchased from Mapeley by Jastar Capital, who have launched consultations for a revised luxury hotel conversion.