The site, 23 km from Valladolid, has been fortified since the tenth century, when it is documented in connection with Moorish forays into the region, under Abd al-Rahman III.

From 1448 to 1452, Portillo was occasionally held by Juan's favourite, Don Álvaro de Luna, although the fortress remained in royal hands.

Don Álvaro, however, fell from favor, and was detained at Burgos, sent to Portillo until two months later he was tried at Valladolid, and subsequently beheaded in the main Plaza on 2 June 1453.

In 1464 Enrique IV of Castile conferred it upon Alfonso, held in trust by Juan Pacheco, the prince's tutor, until his death (1474), though it had been ceded to Rodrigo Alfonso de Pimentel, whose heirs held it until the nineteenth century, when it passed to the family of Osuna.

Juan II of Castille was briefly imprisoned at Portillo in 1444 by the Conde de Castro, escaping by bribing one of his guardians.