A member of the Republican Party, he was elected to a single term as the 27th governor of California from January 6, 1931, until his death on June 2, 1934, at the height of the Great Depression.

Rolph received considerable criticism for publicly praising the citizens of San Jose following the November 1933 lynching of the confessed kidnapper-murderers of Brooke Hart, a local department store heir, while promising to pardon anyone involved, thereby earning the nickname "Governor Lynch.

[6]: 149 After violence erupted during the San Joaquin cotton strike in October 1933, Governor Rolph appointed a fact-finding committee to investigate the deaths of several strikers.

When the committee met in Visalia on October 19, 1933, Caroline Decker, a labor activist who had taken part in other California agricultural actions, took testimony from the strikers who testified about the growers' assaults on striking workers.

After suffering several heart attacks, Rolph died in Santa Clara County on June 2, 1934, aged 64, three years into his term.

On September 7, 1934, the California Toll Bridge Authority passed a resolution "that...the San Francisco-Oakland Bay Bridge be dedicated as a lasting memorial to the memory of James Rolph Jr."[8][9] A 1993 survey of historians, political scientists and urban experts conducted by Melvin G. Holli of the University of Illinois at Chicago ranked Rolph as the twenty-third-best American big-city mayor to have served between the years 1820 and 1993.