Prior to his career as a museum director, Phillips served during World War II, worked as an analyst for the Central Intelligence Agency (CIA) and co-founded a local magazine.

In 1930, the family moved to another house, Dunmarlin, a 17,000 sq ft (1,600 m2) mansion on Foxhall Road NW designed by architect John Russell Pope, to increase the museum's gallery space.

[1] Following his graduation, Phillips attended Yale University, his father's alma mater, for one year before serving in World War II.

He oversaw the expansion of the museum's collection, including key purchases of works by Pierre Bonnard, Henri Matisse, Pablo Picasso and Jackson Pollock.

[8] He "supervised multi-million dollar fundraising campaigns, starting charging admission, established corporate and personal membership programs, and sought arts endowment grants that the museum had seldom if ever pursued.

"[1] Phillips worked for fifteen years to help establish the museum's Center for the Study of Modern Art, "an interdisciplinary forum for scholarly discussion, research, and publication on issues of production.

[1] The couple had previously lived in Georgetown, in a 10,000 sq ft (930 m2) home on O Street NW, after the Phillips estate, Dunmarlin, was purchased by a wealthy Saudi Arabian businessman and demolished.