In May 1866, with the assistance of Thomas Blake Glover, Yoshihara was one of five Satsuma samurai smuggled out of Japan on a Portuguese cargo ship to England, in violation of the national seclusion laws of the Tokugawa shogunate.

The five travelled via the Cape of Good Hope to London, where they met with the members of the first Satsuma delegation to England, which had arrived a year earlier.

In January 1869, Yoshihara was baptized as a Christian at the Owasco　Outlet　Dutch　Reformed　Church shortly before he was accepted into Yale University in New Haven, Connecticut in 1869, where he studied political science and law.

Following the Meiji Restoration, he accompanied Ōyama Iwao, Shinagawa Yajirō and Nakahama Manjirō to Germany in 1871 during the Franco-Prussian War as official military observers, visiting Frankfurt, Berlin and calling on Paris during a ceasefire.

In November 1878, Yoshihara accompanied Matsukata Masayoshi and Aoki Shuzo to Paris in another unsuccessful attempt at revision of the unequal treaties.