Zhao Xi was injured when he fought in the Battle of Kunyang against the Xin dynasty, and later received the title of "Marquis Yonggong" (勇功侯).
He instructed young women to smear dirt and mud on their faces to avoid being violated by the rebels if the latter caught up with them.
In 41 CE, when Zhao Xi was in office as Prefect of Huai County (懷縣), he punished a wealthy landlord Li Zichun and his two grandsons for violating the law.
In 50 CE, the women whom Zhao Xi saved at Wu Pass 25 years ago praised him, so Emperor Guangwu summoned him to the imperial court and commissioned him as Minister Coachman (太僕).
A year later, Zhao Xi was promoted to Grand Commandant (太尉), received the title of a "Secondary Marquis" (關內侯), and was tasked with inspecting the border.
In the spring of 60 CE, Zhao Xi was stripped of his post of Grand Commandant for making an erroneous report on Xue Xiu (薛脩), the chancellor of Zhongshan (中山).
When his mother died of illness, Zhao Xi returned home to mourn, and Emperor Ming attempted to persuade him to remain in court.