The present fortification dates largely to the reconstruction under Ali Pasha in the late Ottoman period, but incorporates also pre-existing Byzantine elements.

Additions in the late 11th century, including the southeastern citadel, are traditionally ascribed to the short-lived occupation of the city by the Normans under the leadership of Bohemond of Taranto.

In the Ottoman era, the city enjoyed considerable prosperity, and reached the height of its pre-eminence under the rule of Ali Pasha, who became the ruler of a large semi-autonomous state encompassing much of modern-day Greece and Albania between 1787 and his downfall and execution in 1822.

[3][5] The northeastern citadel covers an area of approximately 6,000 m2, and is surrounded by a wall that dates partly to the Byzantine period, including the monumental southern gate, flanked by a large circular tower.

[3][7] Outside the citadel but in close proximity lie the Turkish Library, probably attached to the madrasah, an Ottoman bath (hamam) and the Soufari Sarai ("horsemen's palace"), a cavalry barracks built in the last years of Ali Pasha's rule (1815–20).

The palace is described by European travellers and depicted in a print by W. L. Leitch and engraved by H. Adlard, as a large and complex two-storey structure with many windows giving excellent view to Lake Pamvotis.

The palace continued to serve as the city's administrative centre until 1870 when it was torn down, although it had already been badly damaged during the 1821–22 siege by the Sultan's troops that brought about Ali's downfall.

[3][9] Next to the museum, on the easternmost and highest part of the citadel, lies the Fethiye Mosque with the tomb of Ali Pasha and one of his wives to the northwest, covered by an iron lattice work.

[3][11] Other surviving or excavated structures are the kitchens, dating to the early 19th century, located in the northwestern part of the citadel, which now serve as a refectory; the gunpowder store to the northeast of the Fethiye Mosque, which today serves as an educational space, with a ruined building of unknown purpose between it and the kitchens, as well as the ruined base of another large Byzantine-era circular tower; a large two-storey building northeast of the Treasury, possibly a barracks or an unidentified part of the palace, which survives only in half its original length, and now serves as a cultural and exhibition space; and a small bath complex to the north.