The Colley or Cowley family had come to Ireland from Rutland about 1500 and produced a line of Crown servants and soldiers.
Dudley's great-grandfather Sir Henry Colley acquired Carbury Castle in 1554.
Dudley's father acquired substantial lands in County Wexford.
[citation needed] He represented Philipstown[1] in the Irish House of Commons during the Parliament of Charles II.
[citation needed] By his marriage to Anne Warren he was father of Henry Colley[1] and grandfather of Henry Colley, Garret Wesley, and Richard Wesley, 1st Baron Mornington, all of whom also sat in the Irish House of Commons.