[4] After growing up in Hamilton Township and graduating from Trenton Catholic High School, Rafferty enlisted in the United States Navy, serving aboard the USS Randolph for two years.
[4] After law school, Rafferty clerked for U.S. District Court Judge George H. Barlow and worked as a legal analyst for the New Jersey Division of Taxation.
[4][5] In 1971, Rafferty ran for the New Jersey Assembly to represent Hamilton, Trenton, and Washington Township (District 6B).
He and Republican running mate Peter Rossi lost the race for the multi-member district to Hamilton recreation director Francis J. McManimon and incumbent S. Howard Woodson.
[5][4] In 1980, after appeals from John P. Sears, Rafferty agreed to serve as the director of Ronald Reagan's New Jersey campaign.
[8][4] Following the indictment of incumbent U.S. Representative Frank Thompson, Rafferty was approached to run for the seat in 1980, replacing lesser-known Republican nominee Chris Smith.
He hoped to capitalize on his association with now-President Reagan,[8][4] but finished a distant seventh in the Republican primary, which was won by eventual Governor Thomas Kean.
[10] After the primary, Kean considered Rafferty for chairman of the New Jersey Republican Party,[11] but the job was ultimately given to Philip D. Kaltenbacher.