Kozaki was called one of the "Three Elders" of the Kumiai Church alongside Miyagawa Tsuneteru [ja] and Ebina Danjo.

However, in 1876, he was baptized by Leroy Lansing Janes and joined the Kumamoto Band, but kept his Confucianist beliefs.

[1] The Kumamoto Yogakko closed that year, so Kozaki transferred to Doshisha English School, where he met Joseph Hardy Neesima.

In 1886 he published his first major work, the Seikyo Shinron, in which he criticized Confucianism and said that Christianity should be Japan's main religion.

[5] Kozaki returned to his churches in Tokyo, where he continued to write, preach, and conduct missions overseas.