The Magic Castle is a châteauesque residence built in 1909 by real estate investor, lawyer, banker, newspaper editor, and philanthropist, Rollin B.

In addition, "Street lamps along the driveway that once ran along Venice's Victoria Pier, cast iron frieze work from the Masonic temple on Wilshire, dining room paneling from Sunset's demolished Norma Talmadge Building, and chandeliers from the first Bullock's department store (in Downtown)"[6] have also been added.

In his retirement he would sit in the bar area entertaining and teaching, and after his death in 1992 at the age of 98, according to biographer Karl Johnson, he was cremated and a box with his ashes was brought back to the Castle.

[8][9] Many celebrities have performed at the Castle, including Orson Welles, Johnny Carson, Steve Martin, Neil Patrick Harris[10] and actor Jason Alexander.

Magicians Penn Jillette and David Copperfield supported the sale to Pitchford (who married his wife Kristy on stage at the Castle).

'"[15] The lobby of the Castle has no visible doors to the interior, and visitors must say a secret phrase to a sculpture of an owl to gain access, exposing the entrance to the club.

Five magic performances are showcased each night, spread across three theaters: the Close-up Gallery, the Parlour of Prestidigitation, and a large stage in the Palace of Mystery.

Informal performance areas near the five bars give magician members the space for impromptu magic for guests and other patrons.

However, on Saturdays and Sundays, the Castle hosts an "all-ages" brunch and performances which are open to members and their guests, including those under the age of 21.

His wife Geraldine and his eldest son, Bill Jr. continued the publication of the magazine, but the "Academy" lacked any formal structure, and the organization became dormant.

In 1962, William Larsen's younger son, Milt, undertook the job of leasing and restoring an ornately styled, 3-story 1909 house in Hollywood, California and turning it into a meeting place for magicians which he called the "Magic Castle".

As part of their plan to revive the Academy and locate it in the "Magic Castle", Bill Jr. filed articles of incorporation and created a nonprofit corporation, in 1962.

According to the Los Angeles Times the April 2022 purchase of the Castle by businessman Randy Pitchford "is expected to alleviate financial pressure felt by the academy".

[14] To become a regular member, one must first be actively practicing or involved with magic as a career or hobby and audition before the academy's membership reviewing committee.

Honorary and VIP memberships are presented by the board of directors to magicians, celebrities and individual members of the academy who have contributed to the advancement of the art of magic.

[23] Applications and an audition are required, according to Chair Steven Barnes, it is a "'highly competitive program and it's incredibly rare to get accepted on your first try.

General manager Joseph Furlow resigned two weeks after the LA Times investigation of "allegations of sexual misconduct, racism and other issues" in December 2020.

'"[14] Member Chris Hannibal quoted in the Times investigation in 2020 stated that he has witnessed women being treated as "prey or props" and also commented on the interior architecture of the ground-floor bar that is "an open secret" that because of a steep angle women wearing dresses ascending the main staircase "'it is easy to “fully see straight up a dress'".

A sign acknowledging the "ethical concerns" was added but members question the decision to continue to display any of Robinson's images.