The Lytton Savings main branch was later demolished along with a strip mall next to it to make way for a mixed-use development designed by Frank Gehry.

Within a few years they had built and moved into a new house a few blocks east, at 7920 Sunset Boulevard, the site today of the Directors Guild of America headquarters.

Stage and screen actress Alla Nazimova leased Hayvenhurst from William Hay not long after she moved to Los Angeles from New York in 1918.

Faced with a financial crisis in the mid-1920s after her screen career derailed, Nazimova put her property to work generating an income by building a complex of 25 rental "villas" around the original house.

She found her role as a hotel manager unsuitable and discovered that her unscrupulous partners in the enterprise had nearly bankrupted her, so in 1928 she sold out her remaining interest in the property, auctioned off most of her furniture and other household goods, and went back to the Broadway stage.

[6] After Nazimova's renewed Broadway success was cut short by illness in 1938, she returned to Hollywood and rented Villa 24, where she lived until her death in 1945.

[6] Catering to both short-term and long-term guests, the hotel soon gained a reputation as a place where the famous could enjoy living in a quaint, cozy, village-like setting, conveniently located yet shielded from gawking tourists and autograph seekers by discreet security patrols, under a management that was not inclined to probe, judge or interfere with the private — and sometimes public — activities of its often unconventional patrons.

"[8] Fitzgerald's biographer and lover Sheilah Graham later wrote a book about the place, titled simply The Garden of Allah.

[11] Dance band leaders Benny Goodman and Artie Shaw and vocalist Frank Sinatra were among the pop music personalities.

[11] Although celebrities such as Errol Flynn were still calling it home as late as 1957,[12] by that time the hotel's architectural style was long out of fashion and its environs had become more tacky than glamorous.

Lytton's plans for the property included razing the hotel to make way for a new main branch for his bank, which had formerly been headquartered in the Canoga Park neighborhood in the San Fernando Valley.

[16] Within days, all traces of the hotel were gone and construction of the bank building, designed by the Los Angeles-based architect Kurt Meyer in the Brutalist architectural style, had begun.

[18] In response to this, the Los Angeles Conservancy and Friends of Lytton Savings advocacy group waged a preservation campaign to help prevent the building's demolition but ultimately, it was unsuccessful.

For many years, this was on display outside Lytton's building, in a small plaza at the corner of Sunset and Crescent Heights, sheltered from the elements in a sort of shrine with a lofty domed pavilion.

[21][22] The hotel's name was not a direct reference to Islam but rather to Nazimova's first name and the title of a 1905 novel, The Garden of Allah, by British writer Robert S. Hichens.