[4] As an undergraduate at UCLA, Rafferty "took umbrage at many of the things" in the college newspaper, the Daily Bruin, "particularly the editorial page ... to the point of charging into the office and physically attacking me", recalled editor Stanley Rubin in 1970.

[5] In 1937, Rafferty wrote a letter to the Los Angeles Times in which he described The Bruin as "one of the most prejudiced newspapers on the Pacific Coast" and complained that its "radicalism is not so funny if it keeps you from getting a job.

"[6] Rafferty's first job, during World War II, was as a classroom teacher in the Trona Unified School District in the Mojave Desert portion of San Bernardino County, California.

[7] In 1962, he was elected to the nonpartisan office of California education superintendent, defeating Los Angeles school board president Ralph Richardson.

A series of newspaper articles by David Shaw reported that Rafferty had been less than eager to serve his country during World War II after being classified 1A: fit for military service.

[11][12] Two years later, in 1970, Rafferty failed in his bid for a third term as Superintendent of Public Instruction, losing to Wilson Riles, the first African-American to be elected to statewide office in California and a Democrat in the nonpartisan race.

[15] Shortly before his death, he was appointed by President Ronald Reagan to a national advisory board on the financing of elementary and secondary education.

[18] In particular, he wanted schools to focus on phonics, memorization, and drill; use American history and children's classics in teaching from the early grades forward; and drop psychology and "life adjustment" approaches from education.

[21] Rafferty has also espoused hero-worship of the founding fathers in history classes saying “Dwelling on their human faults distorts the overall picture.”[22] Holding the image of the founding fathers even above objectivity: "Balancing virtues with vices, belittling the heroes, dwelling unduly upon the scandals of the past—these are the techniques that produce in the minds of the children a balanced, bland, tasteless, lifeless image of the country, and all in the sacred name of objectivity.”[22]Politically, he was known as an "articulate spokesman for the far right"[23] who had a "nationwide reputation as a Fourth-of-July style orator and writer.