In the face of declining membership, the congregation sold the building in November 1934 to the Soto Zen Mission of the Sokoji Buddhist Church.

The San Francisco Go Club had become the first overseas branch of Japan's Nihon Ki-in in 1936 and in 1937 moved into the left wing of the building.

The San Francisco Zen Center was incorporated separately, comprising the western disciples of Suzuki Roshi, who continued to share the space throughout the 1960s with the Sokoji congregation.

The Sokoji community also moved three years later in 1972, and the building thereafter stood empty and fell into disrepair, coming under the ownership of the San Francisco Redevelopment Agency.

In 1996 the ownership of the building was transferred to the Japanese American Religious Federation and it ultimately was given to Kokoro Assisted Living, a home for senior citizens, most of whom are from the Japanese-American community.