His father Linghu Xi (令狐熙) served as an official during Emperor Wen's reign but died in anger after being imprisoned due to false accusations of corruption.
Late in the reign of Emperor Wen's son Emperor Yang, Linghu Defen was made the magistrate of Yaocheng County (藥城, in modern Bozhou, Anhui), but as by that time the Sui realm was had been thrown into a state of confusion due to agrarian rebellions, Linghu never took office.
In 622, Emperor Gaozu made Linghu Mishu Cheng (秘書丞), the secretary general of the Palace Library, and had him participate on the compilation of the Yiwen Leiju, with the chancellor Chen Shuda.
In 641, Linghu was made a member of the staff of Emperor Taizong's son Li Chengqian the Crown Prince.
In 644, Linghu was recalled to civil service and made the prefect of Ya Prefecture (雅州, roughly modern Ya'an, Sichuan), but was later removed from office due to his acts there.
In 640, Emperor Gaozong put Linghu Defen in charge of editing the laws, and made him the deputy minister of rites again, as well as an imperial scholar at Hongwen Pavilion (弘文館).
As Linghu had compiled the records of Emperor Taizong's reign after 639, he was awarded silk and made an imperial scholar at Chongxian Pavilion (崇賢館).
He was later put in charge of compiling the records of Emperor Gaozong's reign as well, and his title was promoted to Duke of Pengyang.