The Devon historian Tristram Risdon (died 1640), quoting his source "Vincent upon Brooke and Mills", suggested he was lord of the manor of Spencer Combe in the parish of Crediton, Devon, which his ancestor Richard Spencer had inherited by marriage to Alice Hody, daughter of William Hody of Combe Lancells, whose own family had inherited it from the Lancells family.

[4] The American genealogist Douglas Richardson[5] suggests that Sir Robert Spencer was in fact the son and heir of John Spencer, Esquire, MP for Dorset, of Frampton in Dorset, Ashbury in Devon and Brompton Ralph in Somerset, by his wife Jone.

Little if anything is known about the career of Sir Robert Spencer, other than Risdon's statement that he was "Captain of the castle of Homet and Thomeline in Normandy".

[6] Due to his wife's inheritance of the manor and advowson of Hazelbury Bryan in Dorset, Spencer made presentations to the rectory in 1493 and 1496.

[7] He held the following manors, in right of his wife's dower:[8] In about 1465[9] he married (as her 2nd husband) Eleanor Beaufort (1431–1501), the widow of James Butler, 5th Earl of Ormond, 1st Earl of Wiltshire (d.1461) and the daughter and eventual heiress of Edmund Beaufort, 2nd Duke of Somerset (1406–1455), KG, the third surviving son of John Beaufort, 1st Earl of Somerset, the eldest of the four legitimised children of John of Gaunt (1340-1399) (third surviving son of King Edward III) by his mistress Katherine Swynford).