He had a friendly relationship with fellow chancellor Zhang Jiuling, and when another chancellor, Li Linfu, managed to convince Emperor Xuanzong that both Zhang and Pei were engaging in factionalism, both were removed, although Pei continued to serve in important positions in the imperial administration until his death in 743.
He later passed a special imperial examination for boys,[2][3][4] and in his youth became an assistant secretary at the Palace Library.
Emperor Xuanzong was particularly impressed with one of the suggestions, which stated, "If you cause great harm to the people, then you cannot say that the realm is peaceful."
Emperor Xuanzong promoted the three of them—in Pei's case, to be the prefect of the more important Ding Prefecture (定州, roughly modern Baoding, Hebei).
He later successively served as the prefect of Xuan (宣州, roughly modern Xuancheng, Anhui) and Ji (冀州, roughly modern Hengshui, Hebei, note different prefecture than his previous post) Prefectures, before being recalled to Chang'an to serve as the deputy minister of census (戶部侍郎, Hubu Shilang).
Emperor Xuanzong considered going to the eastern capital Luoyang, to decrease the need to requisition food supplies, and he summoned Pei to discuss with him.
Later that year, Pei was made Shizhong (侍中), the head of the examination bureau and a post considered one for a chancellor.
Around the new year 737, Emperor Xuanzong removed Pei and Zhang from their chancellor posts, making them Chengxiang (丞相) -- the heads of the executive bureau (尚書省, Shangshu Sheng) instead.
Later in 737, there was an incident in which the Yang Jun (楊濬) the prefect of Yi Prefecture (夷州, roughly modern Zunyi, Guizhou), was accused of corruption and received a death sentence.
Emperor Xuanzong commuted the sentence to caning for 60 times and exile to Gu Prefecture (古州, roughly modern Lạng Sơn Province, Vietnam).
In 740, Emperor Xuanzong, impressed at the military accomplishments of the general Gai Jiayun (蓋嘉運), the military governor of Hexi (河西, headquartered in modern Wuwei, Gansu) and Longyou (隴右, headquartered in modern Haidong Prefecture, Qinghai) Circuits, summoned him to the capital to reward him and commissioned him to plan an attack on Tufan.
Pei submitted a petition, pointing out that while Gai was brave, he was becoming arrogant and unattentive in light of imperial favors.