The castle was originally Byzantine and was built by Nikephoros II Phokas shortly after 959, it then fell to the Arabs and then to the Seljuk Turks.

The city stands on the northern spur of the Jebel Talat, which forms the east bank of the Orontes with superb views of the Iron Bridge and the Antioch-Aleppo road, making it a high-value position of strategic interest.

[4] After this, the castle remained in Muslim hands for the remainder of the Crusader period, though Count Philip I of Flanders unsuccessfully attempted to recapture it one last time during his pilgrimage to the east in 1177.

[1] In the early 1800s, the Barmada family rebuild Harem[clarification needed] after it was completely destroyed by the Mongols in the 13th century.

[7] The landlords[clarification needed] of Harem between the 1800s until 1963 belonged to four main families: Barmada, Kayali, Al-Kikhia and Hananu.