Designed by prominent Seattle architect John Graham, Sr., the original plans were for a twelve-story building that would be built in two phases but the top 6 floors were never added.

[1] It is located north of historic Pioneer Square, "in the shadow of" the Smith Tower, and adjacent to the former Grand Opera House and the 1910-built Chicago School-style Lyon Building, also designed by John Graham.

Undaunted, he decided to throw his lot in with the growing local lumber industry, landing at Port Gamble, Washington in 1859 where he went to work running logging camps for Cyrus Walker.

The same year he had arrived in Port Gamble, he purchased the first of many pieces of Seattle real estate sight unseen, that by the time of his death would be some of the most valuable property in the city.

After his marriage to Annie Peoples, one of the famous Mercer Girls, in 1867, Brown would make Seattle his permanent home, joining the city council and building a stately residence on his property at First Avenue and Spring Streets in the early 1870s.

Graham's concept sketch was published in the July 9, 1911 edition of the Seattle Times, showing the building as it would have been with the top 2 floors forming the capital of the 3-part design with arched windows and a large dentiled cornice.

[10] Demolition of the old buildings by contractor Sherman G. Combs[8] began in July 1911[7] and excavation of the lot was completed by the end of August.

Brown was removed as receiver from his father's estate by the rest of his family after he was found to have embezzled over $300,000 in funds for his own personal use and to pay off debts for various farming scams.

The Rector Hotel and adjoining Grand Opera House building were foreclosed upon and by 1920 all of Brown's real estate holdings had been sold off to settle their delinquent loans.

The intention was to connect the hotel to the adjacent Opera House building, which had recently been gutted by fire, and add more rooms, but this was never done.