Gao Jifu himself was said to be both studious and trained in martial arts when he was young, and was also said to be particularly appropriate in his actions during the mourning period after his mother's death.
His elder brother, Gao Yuandao (高元道), served as the magistrate of Ji County (汲縣, in modern Xinxiang, Henan).
In 618 or 619, when the Sui Empire was on the verge of total collapse after the death of Emperor Yang, the people of Ji County surrendered to agrarian rebels, and Gao Yuandao was killed in the process.
In 645, during Emperor Taizong's campaign against Goguryeo, Emperor Taizong left Li Zhi at Ding Prefecture (定州, roughly modern Baoding, Hebei), to be in charge of logistics, assisted by a number of officials led by the chancellor Gao Shilian, and Gao Jifu was a member of Li Zhi's staff there.
[4] In 648, he was made Zhongshu Ling (中書令)—the head of the legislative bureau and a post considered one for a chancellor, and also served as minister of civil service affairs.