Chichester was the eldest son of Arthur Chichester, 1st Baron Templemore and Lady Augusta Paget, daughter of Field Marshal Henry Paget, 1st Marquess of Anglesey.

[1] During his military service he gained the rank of Honorary Colonel for the 3rd Battalion, Royal Irish Regiment.

[1] On 3 August 1842, Lord Templemore married, his first cousin once removed, Laura Caroline Jane Paget, daughter of Sir Arthur Paget and Lady Augusta Fane (a daughter of 10th Earl of Westmorland).

[2] Before her death, they had two children:[1] After his first wife's death in December 1871 he married secondly Lady Victoria Elizabeth Ashley, daughter of Anthony Ashley-Cooper, 7th Earl of Shaftesbury and Lady Emily Cowper (a daughter of 5th Earl Cowper), on 8 January 1873 at St George's, Hanover Square, London.

They had one daughter:[1] Lord Templemore died in June 1906, aged 84, and was succeeded by his only son, Arthur.