Governor Lee's administration from 1961 to 1967 saw the establishment of schools, a new airport, roads, Rainmaker Hotel, an educational television system, new harbor facilities, and a fisheries cannery.

That year he was appointed as associate (later becoming deputy) commissioner of the Bureau of Indian Affairs, where he was noted for his skills as a congressional liaison.

[2][4] Lee's entered public service after finishing his degree, and from June 1936 to July 1937, he worked as an economist with the United States Department of Agriculture's Resettlement Administration in Moscow, Idaho.

[3] Lee was a consultant on loan to the United Nations in 1949, and spent three months travelling in the Near East conferring with Arab and Israeli leaders to assist refugees displaced by the Arab–Israeli conflict.

Dillon S. Myer, the former director of the War Relocation Authority, had been placed in charge of the Arab Refugee program, and had asked that Lee was assigned to assist him.

[8] It was Carver who suggested a method of selecting the next Governor of American Samoa that avoided accusations of political bias.

Delma H. Nucker was vetoed as a suggestion for being "too Eisenhower", Hillary A. Tolson, the deputy director of the Park Service, turned it down, and it was only after "much arm twisting" that Lee accepted the role.

Concurrently in 1961, US Air Force Major Eric J. Scanlan, a native Samoan, was appointed as government secretary, a role similar to lieutenant governor.

[9] Stewart Udall, the United States Secretary of the Interior, said that Lee was appointed as Governor due to his "unique experience and long familiarity in helping to solve" socio-economic problems similar to the ones American Samoa faced.

On June 21, 1963, Paramount Chief Tuli Le’iato of Faga’itua was sworn in and installed as the first Secretary of Samoan Affairs by Lee.

[9] Lee received the President's Award for Distinguished Federal Civilian Service from Lyndon B. Johnson in 1966 who also credited him with turning American Samoa from a "Pacific Slum" to a "showplace for progress".

[2] He was instrumental in successfully bringing American Samoa's plight to the attention of Congress, and in reforming the infrastructure and educational system.

Just over a year later, President Johnson appointed Lee to a seven-year term as a commissioner on the Federal Communications Commission (FCC), which began on October 28, 1968.

[17] During his time on the commission, he frequently joined Nicholas Johnson in voting against station purchases in order to avoid media concentration.

[3] In 1963, William C. Brown Jr. (“Puka Bill”) was arrested for threatening to kill Governor Lee and detained for over 48 hours without being informed of the specific charges.