He was the eldest son and heir of Sir Geoffrey Pole and Constance Packenham who had some claim to the English throne as a Plantagenet descendant of George, Duke of Clarence.
After the fall of his uncle, Henry Pole, Baron Montagu and his grandmother, Margaret, Countess of Salisbury, Arthur was imprisoned for a time.
He was imprisoned with his brothers, Edmund and Geoffrey, in the Tower of London from October 1562 or 1563 for conspiring to advance his own or Mary, Queen of Scots' claims to the throne of England.
[3] Arthur was imprisoned in the Beauchamp Tower, where an inscription can be found that reads: ‘Deo Servire / Penitentiam Inire / Fato Obedire / Regnare Est / A Poole / 1564 / IHS’ (‘To be subject to God, to enter upon penance, to be obedient to fate, is to reign, A Poole, 1564, Jesus’).
[1] Between 15 September 1562 and 27 January 1563, he married Mary Holland, who died before 16 November 1570, daughter of Sir Richard Holland of Denton, Lancashire, and wife Eleanor Harbottle, who was the widow of Sir Thomas Percy (attainted and executed in 1537), son of Henry Algernon Percy, 5th Earl of Northumberland and Catherine Spencer.