It is possible that she dropped the use of her cognomen after the damnatio memoriae of her paternal aunt Livilla (sister of Germanicus and Claudius) after whom she was named.

She was born on Lesbos, one of the many Greek islands during her parents' grand tour of the eastern Mediterranean, leading Germanicus to his command base in the imperial province of Syria for the maius imperium given to him by Tiberius over the territory east of the Adriatic Sea.

As a young child, she was with her mother and brother Caligula when they returned to Rome after Germanicus' untimely death in Antioch in 19 AD.

[3] During the first years of Caligula's reign, Livilla, along with her elder sisters Agrippina the Younger and Julia Drusilla, received considerable honours and striking privileges, such as the rights of the Vestal Virgins (like the freedom to view public games from the upper seats in the stadium), the inclusion of her name in the oath of loyalty to the emperor and her depiction on coins.

After the deaths of Caligula, his fourth wife Milonia Caesonia, and their daughter Julia Drusilla, she returned from exile on the orders of the new emperor, Livilla's paternal uncle Claudius.

[7] Political considerations may have played a role in Julia Livilla's fate, more than just moral or domestic preoccupations as inferred in the ancient sources.