His maternal grandmother ran a boarding house in Tombstone and fought to keep the Tucson public schools desegregated.
When Whites established separate schools, she relocated to Los Angeles, having decided that "she was not going to bring those kids up in a segregated environment.
They joined the rest of the family in Los Angeles in 1915, living for a time in a large house on Burlington Avenue.
There, his paternal grandmother's sister, who was a songwriter, poet, and painter, helped Delilah L. Beasley write her 1916 history The Negro Trail Blazers of California.
His mother and stepfather had children of their own, and Williams paid little attention to the Rumford brothers, who had to work to survive.
He was accepted to the school of pharmacy at the University of California, San Francisco, and worked as a parking valet and a doorman at night.
Rumford appealed on the grounds that the board had publicized the statistic that African Americans suffered from sexually transmitted diseases at a greater rate than other ethnic groups, but had not taken the opportunity to do something about it.
There he worked under Dr. Benjamin Black, a former army doctor, "outstanding authorit[y] in hospital administration", and "a very prejudiced person.
"[1]: 9–10 Black was hesitant to hire Rumford, suspecting him of wanting to "make trouble", and was reluctant to raise his salary.
But with the help of influential friends, including Thomas E. Caldecott and county Supervisor Harry Bartell, he received a raise at the beginning of World War II.
For about a year, he worked primarily at army bases, such as Camp Knight, where his main responsibility was apprehending carriers of STDs.
It worked with the Committee on Fair Play to fight the Japanese American internment, and helped the City establish a Human Relations Commission.
He had been involved with the Appomattox Club, "perhaps one of the first" African American political organizations in the Bay Area, since 1932, when it supported the policies of the Roosevelt administration.
While Rumford was in the legislature, his Berkeley pharmacy "became an informal headquarters for other emerging politicians, such as future city of Oakland mayor Lionel Wilson and California Supreme Court Justice Allen Broussard.
It passed the assembly in 1955 with the support of Democrats, some of whom were personally opposed to it but succumbed to pressure from within the party, and even a few Republicans, "because they thought it was basically and morally right.
"[1]: 50 It was signed by incoming Governor Pat Brown in 1959, and the Fair Employment Practices Commission was set up the same year.
Rumford served on the assembly's Civil Service Commission, which addressed issues of discrimination in government employment, including the California Highway Patrol and the State Department of Public Health.
The bill was at the top of Governor Brown's legislative agenda, and it had been endorsed by the NAACP and the California Democratic Party.
"[1]: 92–93 John Holmdahl, the current senator, announced that he would not run to keep his seat, which eased the tension between Rumford and Nicholas Petris, the other contender.
Petris's law partner, Edward Fitzsimmons, filed for candidacy, as did Superior Court Judge Victor Wagner and another candidate.
Rumford took the case to Alameda County district attorney Frank Coakley, who found large discrepancies between the registrar's records and those kept by the precincts.
Door-to-door studies found that "there had been no purging of the county list for years and that thousands of sample ballots were lodged in the post offices, both in Oakland and Berkeley.
In 1970, he was selected to serve on the Grand Jury, and volunteered for the Committee on Courts Enforcement, which looked into the electoral system in Alameda County.
While the commission did not open the case of Rumford's senate run, it did make inquiries into the registration process and as to why so many people were ineligible to vote, as well as looking into the reasons for the quantities of ballots at the post office.
He served for five years as assistant director for Consumer Protection and State-Federal Relations of the Federal Trade Commission (FTC).