The Birges had two children, Carolyn Elizabeth (Mrs. E. D. Yocky) and Robert Walsh,[2] Associate Director of the Lawrence Berkeley National Laboratory in 1973-1981.

Birge was unafraid of scientific controversy and persevered with his course on atomic structure, attracting future Nobel Laureates in chemistry William Francis Giauque and Harold Clayton Urey.

There ensued a frustration with the conventional methods of statistics as applied in physical science which led to a collaboration with W. Edwards Deming.

In the late 1930s, Birge opposed the creation of an assistant professorship for Oppenheimer's associate Robert Serber, writing another colleague that "one Jew in the department is enough.

Though an outspoken critic of the oath,[citation needed] after much searching of conscience, Birge decided that his loyalty to the department and the university demanded that he sign and fight from within for freedom of speech.