Although he only belonged to a side branch of the more prestigious Ise family, he fought his way up, gaining territory and changing his name in imitation of the illustrious Hōjō.

[2] His sister was married to Imagawa Yoshitada, a major daimyō from a prestigious cadet branch of the Ashikaga family.

About 1475, under the cognomen of Ise Shinkuro, he worked for Imagawa, the constable of Suruga Province, and eventually became an "independent leader" with a number of warriors joining him.

In 1491, he was able to take Horigoye after the death of Kantō kubō Ashikaga Masatomo, gaining control of Izu Province.

He gained control of Izu Province in 1493, avenging a wrong committed by a member of the Ashikaga family which held the shogunate.