In 1660, this clique of vassals and kinsman made a successful appeal to the rōjū to have Tsunemune removed from office and placed under house arrest in Edo under the charges of public drunkenness and debauchery while he was visiting the capital on sankin-kōtai and to supervise corvée labor on a canal.

According to one of the most popular legends, the scheming Date Munekatsu took the impressionable young Tsunamune to the licensed quarters of Yoshiwara, where he passionately fell in love with a prostitute, Takao.

[3] While Tsunamune remained under house arrest, the domain suffered greatly under the corrupt rule of Date Munekatsu and Tamura Muneyoshi.

After ten years of violence and conflict, Aki Muneshige, a relative from a cadet branch of the Date clan and retainer of Tsunamune managed to register a complaint to the shogunate officials about the mismanagement of the domain.

As for Tsunamune, he lived the next 50 years in loose house arrest at the clan residence in the Ōi area of Edo, devoting his time to the arts, studying painting under Kanō Tan'yū, calligraphy, waka poetry, Maki-e lacquerware and even learning to forge Japanese swords.

His body was so well preserved that an autopsy could be performed, revealing that he had a height of 158 cm and blood type A+, and had died of oral cancer.