The first 11 sailors were found on Tori-shima, where Manhattan anchored to hunt for turtles to supplement the ship's provisions, and were survivors from the Koho-maru wrecked en route to Edo.
[1] The next day 11 more sailors were found on a foundering Japanese boat Senju-maru (along with a detailed navigation map of Japan).
Outside Edo Bay four of the survivors took a Japanese boat with a message that Cooper wanted to deliver the remainder to the harbor.
[4] The Japanese normally wanted to avoid contact with outsiders due to the Tokugawa shogunate's official policy of national isolation.
He turned it over to the United States government when the ship returned to Sag Harbor on October 14, 1846.