Pelkhor-tsen, son of Langdarma (anti Buddhist King of West Tibet) after whom the monastery is named as Pelkor Chode, lived here and attempted to perpetuate the Yarlung dynasty of his father who had been assassinated.

[6] The Tsuklakhang, the main temple of the monastery was built in 1418–1428 by Rabten Kunzang Phak, the second Prince of Gyantse, who was a devotee of Kedrub Je (1385–1438), one of Tsongkapa's leading disciples later recognized as the 1st Panchen Lama.

[2][5] Another testament to Prince Rabten Kunzang Phak's period is the public display of two gigantic paintings; (Thangkas) of Shakyamuni Buddha flanked by his two principal disciples, of Maitreya, Manjushri and many more on the occasion of the Gyantse festival that is held in the fourth lunar month of the Tibetan calendar.

After the town was occupied, several British officers visited the monastery and seized several statues and scrolls, some of which were subsequently brought back to the United Kingdom.

[3][4] The most striking architecture in the complex, a symbol of Gyantse, is the Bodhi Dagoba (Tibetan name: Pelkhor Choede), popularly called as the 'Kumbum'.

The chapels have the finest display of Tibetan art in "vibrant colour and naturalistic style"; in the faces of the murals Chinese images are discerned.

The 76 chapels have images that form "a progressive hierarchy of three-dimensional manadalas, as outlined in the Sakyapa compilation known as the Drubtob Gyatsa, ensuring that the stupa encapsulates within it the entire spiritual path and gradation of the tantras".

The impressive 48 pillared hall is decorated with numerous silk Tankhas, images of Sakya protectors, frescoes of frightening scenes of charnel grounds and several original paintings and sculptures, in 15th century Tibetan style.

In the Vajradhatu Chapel (Dorje Ying Lhakhang) on the west, Sarvavid Vairochana's statue made in clay is deified and is surrounded by four meditational Buddhas.

Images of Atisha, Kamalashila, Padmasambhava, Shantarakshita, Manjushri, eleven-faced Avalokiteshwara, Vajrapani and Shakyashri of Kashmir are also seen in this chapel.

The south wall of this chapel depicts a Reliquary of Prince Rabten Kunzang Phak (founder of the temple) in a recessed chamber and also many volumes of canonical texts.

[17] On the top floor, is the Zhalyekhang chapel, which has 15 mandalas of 8 metres (26 ft) diameter painted on the walls, associated with meditational deities.

Images of Jowo Shakyamuni, Maitreya, Manjushri, Tsongkhapa with his disciples, Amitayus, Tara, Sitatapatra and Padmasambhava are depicted.

The fortress is dated to 1268 and a castle was also built inside by local Prince Phakpa Pelzangpo (1318–1370) who was influential with the Sakyapa overlords.

Buddhist guru Butan Rinchen Drub of Zhalu resided here at the invitation of the prince and made it his religious seat.

Later, in the 14th century, the palace was moved from the fort to the Gyantse town where Kunga Phakpa had built a larger complex of buildings and monasteries.