He served during the administrations of U.S. presidents Harry S. Truman, Dwight D. Eisenhower, John F. Kennedy, Lyndon B. Johnson, and Richard M. Nixon.

A proponent of free markets, he objected to President Truman's continuation of many New Deal and World War II policies.

From 1947 through 1948, Williams worked to root out corruption in the Internal Revenue Service, exposing the illegal activities of two hundred employees of the Treasury Department.

[11] Williams was the distinctive 67th vote in favor of ending the filibuster of the 1964 Civil Rights Act, leading Mike Mansfield to proclaim "That's it!".

[12] In 1967, Williams helped defeat a proposed rule change that would have eliminated the filibuster, a tool that had been of great use to him in exposing government waste and misconduct.

Williams, as well as his Senate colleague Prescott Bush of Connecticut, was considered a possible running mate for Republican presidential nominee Dwight D. Eisenhower in 1952, but removed himself from consideration.

He was also considered for a spot on the Republican ticket in 1964 and as a possible replacement for Spiro Agnew when he resigned as vice president of the United States in 1973.

In September 1966, Williams assailed the anti-inflation program of the Johnson administration as a "piece-meal approach" to a larger issue and advocated for a five percent across the board tax hike as well as Congress resuming a leadership role on the subject of enacting "necessary remedies to stave off financial collapse that may engulf us".