It was probably acquired by the Farnese family as a result of confiscations early in the 17th century.
The work was inherited by Leopold, Prince of Salerno, son of Ferdinand I of the Two Sicilies.
In 1854 Leopold's collection was sold in its entirety to his nephew Henri d'Orleans, Duke of Aumale, then in exile in Britain - Portrait was then still attributed to Romano.
The current attribution was first mooted in 1932 by Bernard Berenson, though Felton Gibbons's monograph on the painter preferred to see it as a studio copy after a lost autograph original - Gibbons also argued it was a pendant to Portrait of a Man (Fitzwilliam Museum).
The treatment of costume details is also reminiscent of the women in Dossi's Allegory of Hercules (Uffizi) and Portrait of a Man (Louvre).