After the end of World War II, his family moved to Kyushu, Japan, and the young Victor was sent to live with relatives in Tokyo.
There he started his martial career by joining the amateur wrestling club of Nihon University, where he was studying medicine.
[1] In 1965, becoming interested in the Russian martial art of sambo, Koga associated with fellow judoka and wrestler Ichiro Hatta in order to pioneer it in Japan.
After forming the Japanese Sambo Federation, Koga moved to Soviet Union to learn personally the style.
[1] One of his most famous students in Japan was Satoru Sayama, who went to open the first mixed martial arts promotion in the form of Shooto.