The area is one of the most popular attractions in the capital, and can be accessed by bus or in the carriages of a funicular (Budavári sikló).

King Béla IV built the first fort on Castle Hill between 1247 and 1265 after a first Mongol invasion of Hungary in 1242.

Prince Stephen's brother, who later became King Louis I, relocated his seat from Visegrád to Buda in 1347 and began construction of the royal palace and its defense system, which lasted two centuries.

Construction on the Fresh Palace (Hungarian: Friss-palota) began in the 1410s and was largely finished in the 1420s, although some minor works continued until the death of Sigismund in 1437.

During the first decades of his reign he finished the work on the Gothic palace, and the Royal Chapel, with the surviving Lower Church.

The medieval palace was destroyed in the great siege of 1686 when Buda was captured by allied Christian forces.