He was the son of Charles Cecil Byers (1888–1957), a Lloyd's underwriter, who was Liberal candidate for Westbury at the 1935 general election.

He moved with the family to Potters Bar and was educated at Westminster School, followed by Christ Church, Oxford, where he won a Blue for athletics.

[1] While at the University of Oxford, where he gained his degree in Philosophy, Politics and Economics, he met Joan Oliver, whom he married in 1939.

He was mentioned in dispatches three times, was created a Chevalier of the Legion of Honour, and was awarded the Croix de Guerre.

[3] In the 1945 general election, Byers gained the formerly Conservative seat of North Dorset, with the absence of a Labour candidate being a key factor in this success.