Originally constructed for Heshen, an official renowned for being the most corrupt official in Chinese history, it was later renamed after Prince Gong, a Manchu prince and influential statesman of the late Qing dynasty, who inhabited the mansion in the late 19th century.

In 1921, after the collapse of the Qing dynasty, Prince Gong's grandson, Puwei, offered the property as a mortgage to the Order of Saint Benedict of the Catholic Church.

During the Cultural Revolution, the mansion was used by the Beijing Air Conditioning Factory until it experienced a revival in the 1980s.

In 1982, it was declared a Major Historical and Cultural Site Protected at the National Level in Beijing.

In addition to the mansion, there is a 28,000-meter (92,000 ft) garden with 20 scenic spots, pavilions, artificial hills including rock originating from the Lake Tai in Jiangsu, and ponds.

There is an eight-meter-long stele which bears the Chinese character 福 (fú: lit "fortune") based on the calligraphy of the Kangxi Emperor.