In 1623, his father fell out of favor with Shōgun Tokugawa Iemitsu, and was exiled to Kōza District in Sagami Province.

In 1634, Munetoshi received permission to return to Edo, and on December 1, 1638, was appointed to the minor post of Shoinbangashira, a hatamoto-level position with revenue of only 3000 koku.

On May 23, 1644, he was promoted to Ōbangashira, and by January 19, 1648, received an additional 27,000 koku, which made him daimyō of the newly created Komoro Domain in Shinano Province.

In order to take up his posting to Osaka, he surrendered Komoro Domain back to the shogunate, in exchange for 20,000 koku of additional territories scattered in Settsu, Kawachi and Izumi, Tōtōmi, Musashi and Sagami Provinces.

On August 18, 1678, he retired from his position of Osaka-jō dai and was assigned to Hamamatsu Domain in its place, which he ruled to his death on March 16, 1679.