Sir Charles married Mary, the youngest daughter of George Eastwood of Thornhill in the West Riding of Yorkshire, and had four children: Charles (who became Lancaster Herald of Arms in Ordinary), William, John and Mary.
There was opposition to his obtaining the position because of his relation to the Townleys of Lancashire, who were involved in the Jacobite rising of 1715.
The king, George II, eventually granted letters patent appointing him to the position.
Townley's appointment was also notable for having a warrant issued by Francis Howard, Earl of Effingham as Deputy Earl Marshal allowing the ceremony of creation as a herald to be dispensed with, even though a patent had been signed to perform it.
He was knighted in September 1761 and ended his career with an appointment to the most senior position of Garter Principal King of Arms in 1772.