He is described by Dugdale as "a person comely of stature, high of courage and conformity of disposition to King Henry VIII, with whom he became a great favourite."

Brandon held a succession of offices in the royal household, becoming Master of the Horse in 1513, and received many valuable grants of land.

At this time, Henry VIII was secretly urging Margaret to marry Lisle, whom he created Duke of Suffolk on 4 March 1514, although he was careful to disclaim any complicity in the project to her father, Maximilian I, Holy Roman Emperor.

[10] Brandon was appointed Earl Marshal of England in 1524, a position previously held by Thomas Howard, 2nd Duke of Norfolk.

However, in 1533 he relinquished the office to Thomas Howard, 3rd Duke of Norfolk, "whose auncesto[ur]s of longe tyme hadde the same until nowe of late.

He was sent with Thomas Howard, 3rd Duke of Norfolk, to demand the Great Seal from Wolsey; and Suffolk acted as High Steward at the new queen's coronation.

[10] His family had a residence on the west side of Borough High Street, London, for at least half a century prior to his building of Suffolk Place at the site.

Love between Charles and the young Dowager Queen Mary had existed before her marriage, and King Francis I roundly charged him with an intention to marry her.

Anne Browne died in 1511, but Margaret Neville, from whom he had obtained a declaration of nullity on the ground of consanguinity, was still living.

[18] Before 7 February 1507, Charles Brandon firstly married Margaret Neville (born 1466), widow of Sir John Mortimer (d. before 12 November 1504),[19][20][21] and daughter of John Neville, 1st Marquess of Montagu (slain at the Battle of Barnet) by Isabel Ingaldesthorpe (or Ingoldesthorpe), daughter and heiress of Sir Edmund Ingaldesthorpe (or Ingoldesthorpe) and his wife, Joanna Tiptoft.

[5][22] In early 1508, in a secret ceremony at Stepney, and later publicly at St Michael's, Cornhill,[5] Charles secondly married Anne Browne (the step-daughter of Margaret Neville's sister, Lucy Neville), daughter of Sir Anthony Browne (Standard Bearer of England in 1485), by his first wife, Eleanor Ughtred,[23] the daughter of Sir Robert Ughtred (c. 1428 – c. 1487) of Kexby, North Yorkshire[24] and Katherine Eure, daughter of Sir William Eure of Stokesley, Yorkshire.

[citation needed] In May 1515, Charles thirdly married Mary Tudor, Queen Dowager of France (18 March 1496 – 25 June 1533).

They had two sons who died young, and two daughters: On 7 September 1533, hardly two months after Mary's death, Brandon married for the fourth time.