In 659, Zhangsun Wuji was falsely accused of treason by Empress Wu's political ally, Xu Jingzong, and eventually ordered to be sent into exile by Emperor Gaozong.

Xu Jingzong subsequently sent the official Yuan Gongyu (袁公瑜) to force Zhangsun Wuji to commit suicide on his way to exile.

It was written that his ancestors traced their origin to the Xianbei dynasty Northern Wei's founding emperor Emperor Taiwu of Northern Wei's 17th generation ancestor Tuoba Kuaili (拓拔儈立)—that their ancestor was Tuoba Kuali's third son, who took the surname Baba (拔拔), eventually changed to Zhangsun when Emperor Xiaowen of Northern Wei changed Xianbei surnames to Han surnames in 496.

Late in 626, when Emperor Taizong personally ranked the contributions of the generals and officials in order to grant them fiefs, Emperor Taizong ranked five of them—Zhangsun Wuji, Fang Xuanling, Du Ruhui, Yuchi Gong, and Hou Junji to be contributors of the highest grade, and Zhangsun was created the Duke of Qi.

Later that year, when Emperor Taizong heard that Eastern Tujue's Jiali Khan Ashina Duobi, who had made a deep incursion to Chang'an when Emperor Taizong first took the throne in 626, was having internal problems with his subordinates, he consulted Zhangsun and Xiao Yu, asking for their opinions.

Xiao advocated an attack on Eastern Tujue, but Zhangsun pointed out that the peace treaty between the two states should not be breached, and Emperor Taizong accepted his suggestion.

In 637, a major revision of the Sui penal laws, led by Fang but assisted by Zhangsun, was completed, with 500 sections dividing the punishments into 20 grades.

Also in 637, as part of Emperor Taizong's scheme to bestow prefectures on his relatives and great generals and officials as their permanent domains, Zhangsun's title was changed to Duke of Zhao, and he was given the post of prefect of Zhao Prefecture (趙州, roughly modern Shijiazhuang, Hebei), to be inherited by his heirs.

Many officials opposed the plan, with the strongest opposition coming from Zhangsun, who also had his daughter-in-law, Princess Changle (Emperor Taizong's daughter) submit oppositions on his behalf as well, and Emperor Taizong cancelled the scheme, although Zhangsun's title remained Duke of Zhao.

Emperor Taizong put the investigation in the hands of Zhangsun, Fang Xuanling, Xiao Yu, Li Shiji, and the officials in charge of the supreme court and the legislative and examination bureaus of government, and they determined that, indeed, Li Chengqian had plotted to overthrow Emperor Taizong.

In 644, when Emperor Taizong, at an imperial gathering, stated to his key officials their strengths and weaknesses, he spoke, with regard to Zhangsun: Later in 644, when Emperor Taizong launched a major attack against Goguryeo, he had the generals Li Shiji and Li Daozong lead the forward forces, while commanding the main forces himself, assisted by Zhangsun, Cen, and Yang Shidao.

He then considered directly attacking Goguryeo's capital Pyongyang itself, but Li Shiji believed that if Ansi (安市, in modern Anshan, Liaoning) was not captured first, the general in command of Ansi (a capable general known in Korean popular legends as Yang Manchun, although whether that was his real name is unknown), might attack Tang forces from the rear.

Zhangsun opposed, believing the strategy to be too risky without first capturing Ansi and Geonan (建安, in modern Yingkou, Liaoning) first.

In 647, when Zhangsun's uncle Gao Shilian died, Emperor Taizong, who had just recently himself recovered from an illness, wanted to attend Gao's wake, but Zhangsun lay down in the path of his horse, blocking him, reasoning that as someone who had recently recovered, it was inappropriate for Emperor Taizong to attend the wake.

In fall 647, when a commoner named Duan Zhichong (段志沖) submitted a petition to Emperor Taizong, asking him to yield the throne to Li Zhi, Li Zhi became concerned that Emperor Taizong would suspect that the proposal came from him, and Zhangsun requested that Duan be executed.

He soon died, and by Zhangsun's orders, his death was kept a secret until his casket, accompanied by Li Zhi, was returned to Chang'an.

Emperor Gaozong also gave him the chancellor de facto designation of Tong Zhongshu Menxia Sanpin (同中書門下三品).

In 652, with Emperor Gaozong's wife Empress Wang being sonless, her uncle Liu Shi, also a chancellor, suggested to her that she propose that Emperor Gaozong's oldest son, Li Zhong, whose mother Consort Liu was of low birth, be made crown prince, believing that Li Zhong would be grateful to her.

Zhangsun, however, used this opportunity to carry out a purge—and in spring 653, Zhangsun persuaded Emperor Gaozong to issue an edict executing Fang Yi'ai, Xue, and Chai, while ordering Li Yuanjing, Li Ke, and Princesses Gaoyang and Baling (Chai's wife) to commit suicide.

(For these actions, he was heavily criticized by Liu Xu, the lead editor of the Old Book of Tang, who commented that perhaps it was karmic that eventually Zhangsun himself would be falsely accused.

In fall 655, after an imperial gathering, Emperor Gaozong summoned Zhangsun, Li Ji (i.e., Li Shiji—he had dropped the character "Shi" by this point by Emperor Gaozong's orders that naming taboo was to be observed as to the characters Shi and Min, not just the sequence "Shimin" as Emperor Taizong had ordered), Chu, and Yu Zhining to the palace—an order that Chu correctly guessed that was intended to call them into a meeting in which he would try to persuade them to agree with his desire to replace Empress Wang with Consort Wu.

When Zhangsun, Chu, and Yu attended the meeting, Emperor Gaozong did indeed propose to replace Empress Wang with Consort Wu.

By the beginning of 657, the power of Empress Wu and her allies was so great that they began to retaliate hard against the authorities who opposed them.

Xu thereafter, when investigating a report of factionalism by the low level officials Wei Jifang (韋季方) and Li Chao (李巢), manufactured evidence that Zhangsun had plotted treason with them.

(As part of the same reprisal transactions, Empress Wang's uncle Liu Shi was also executed in exile.