The Chinese Culture Center leases approximately 20,000 sq ft (1,900 m2) within the building for rotating exhibitions at a nominal cost due to lobbying from the local Chinese-American community.

[11] The building was completed by September 1900;[13] shortly thereafter its proximity to Chinatown was questioned, as "city officials and business men" found it "very obnoxious ... to have the Chinese quarters so near the new hall of justice.

The San Francisco Planning Commission authorized the sale of the old Hall of Justice site in December 1960 and set an August 1961 move-out date for the final agencies occupying it.

SFGCCSA hired J. Francis Ward and Associates in April 1964; that firm had previously designed the Ping Yuen public housing project in Chinatown, and assigned the redesign work to Thomas Hsieh.

[23]: 7 The site was planned to be sold to private interests at a purchase price of US$875,000 (equivalent to $8,460,000 in 2023); prospective bidders were required to pay an $85,000 deposit, with bids due no later than June 1, 1965.

[26] Businesses interested in the development included the Sheraton Hotels and Resorts chain and Tishman Realty & Construction (supporting the Chen/Cherk design), Cahill Contractors (Justice Enterprises), and Greyhound Lines (Haas & Haynie).

[21][28]: 107  In early November, the SFBOS approved the creation of a Chinese cultural center within the old Hall of Justice site, with a minimum of 20,000 sq ft (1,900 m2) of floor space to be dedicated for that purpose.

[29] Later that month, Harold Moose, president of Justice Enterprises, declared their proposal would meet the minimum floor space requirement at an annual rental cost of $5,000.

[39] It has since been named eligible for listing on the California Register of Historical Resources in 2021, both as an exceptional example of Brutalist architecture and as a symbol of the growing political power of the Chinatown community.

The ceremonies were attended by Mayor Joseph Alioto; Herman announced the National Palace Museum in Taiwan would make a major donation to the Chinese Culture Center within the hotel.

[23]: 10–13  The Chinese-American community of northern California raised $560,000 to complete the Cultural Center, which began final construction in late January 1973[49] and held a gala opening on October 18, 1973.

[23][51] Exterior updates in 1991 include modifications to the first-floor windows on the Washington Street facade, a ramp to accommodate patrons under the Americans with Disabilities Act, new lighting, and ten new flag poles.

[59] Final designs announced in December included a tile mosaic entitled "Sunrise" by Mik Gaspay to be installed at the eastern end next to the hotel.

[61][62] The San Francisco Department of Recreation & Parks solicited community input for a planned renovation of Portsmouth Square in 2017 and 2018;[63] by a margin of more than three to one, the attendees recommended removing the bridge altogether.

[40]: 8 Longtime San Francisco Chronicle architecture critic Allan Temko called the Holiday Inn a "bulbous slab" that "exhibits just about every mistake that can be committed in urban design"; urban design critic John King added it was "the most unsociable chunk of concrete in San Francisco ... grim on the skyline and worse on the ground ... the harsh tower standing aloof from the sullied street".

[68] Alex Lehnerer noted the Holiday Inn was one of four high-rises within San Francisco, all built in the late 1960s and early 1970s, which attracted disproportionate criticism; the other three were the Bank of America Building (1969, "too big and too dark"), Embarcadero Center (1971–82, "too large and too bulky"), and the Transamerica Pyramid (1972, "too bizarre").

[71] The new Holiday Inn appeared in the opening scenes of the 1971 film Dirty Harry, in which "Scorpio" shoots from 555 California and kills a swimmer at the now-closed pool on the building's roof.