The house owed its various names to the original owner, Senator William Morris Stewart, the imposing, turreted facade, and its prominence in an area considered undesirable at the time of its construction.

[1][2] Hillyer and Sunderland encouraged Stewart to build a large house in Pacific Circle to spur further development and increase property values in the neighborhood.

Despite financial difficulties that he was experiencing at the time, Stewart began plans for his house and selected architect Adolf Cluss (1825-1905) to design it.

[2][3][4] The design of the nearly 19,000 sq ft (1,800 m2) house was influenced by the pentagonally shaped, 16,322 sq ft (1,516.4 m2) lot on the north side of the arc of the circle between Connecticut Avenue and Massachusetts Avenue[5][6][7] All of the house's furnishings were purchased in 1872 by Stewart's wife, Annie, daughter of Senator Henry S. Foote, during a visit to France.

[5][8][9] There was mixed reaction upon completion of the five-story, Second Empire house, the first to be built on the circle and considered to be one of the most elaborate private residences in the city.

The upkeep of such a large house, including a stable with thoroughbred horses and paying many servants, was financially draining.

[1][5] According to one newspaper reporter, "attending a reception at Stewart's Castle makes one feel like Marco Polo at the Court of Kublai Khan.

"[12] In 1874, the ornate British Legation was built one block south of the circle and the area became a more acceptable section of the city according to prominent members of society.

After only two years in his new house and having exhausted most of his funds, in 1875, Stewart and his family moved back to California, where he resumed his law practice and mining operations.

[1] By 1879, Annie Stewart had grown tired of living in California and returned to their home in Washington, D.C.[1] She made plans to host a New Year's Eve reception at Stewart's Castle, but on the night of December 30, while Annie was visiting friends, a fire broke out due to a defective flue.

[1][13] The house was abandoned and sat vacant until 1883 when Stewart hired architect Robert Isaac Fleming to oversee the necessary extensive repairs.

Nearby residents found the activities of the Chinese diplomatic corps to be peculiar, including staff doing laundry in what was now called Dupont Circle and playing hide-and-seek at night.

[16][17] A local newspaper reported one such event: "Washington ladies are delighted to learn that some of their number have been received by the wife of the new Chinese Minister.

She greeted them in her boudoir on the second floor of Stewart's Castle, toddling toward them on her three-inch-long bound feet, and clad in handsome silken robes of pink, lavender, and blue.

[19] The family continued living in the house until September 1899 when it was sold to wealthy Montana Senator William A. Clark for an estimated $145,000.

"[24] Initial plans for the Stewart Castle site were for a large apartment building or a twelve-story hotel to be erected on the lot.

[1] For more than 20 years, the site where Stewart's Castle once stood remained a vacant lot overgrown with weeds.

[1][28][29] In 1990, Riggs Bank announced it would tear down the branch and construct a 10-story, 100 ft (30 m) tall office building on the entire block, but opposition by neighborhood activists and historic preservationists derailed the project.

[30][31] In 2015, PNC sold the property to L&B Realty Advisors LLP for $60.75 million, but signed a 15-year lease to continue operating its banking branch.