James was born into the Óvár branch of the gens (clan) Győr of German origin, as one of the two sons of Conrad I, Master of the cupbearers.
[1] He is first mentioned by contemporary records in 1282, when his long-lived father Conrad was already expelled from his landholdings in Northwest Hungary by the powerful Kőszegi family.
[4] As the Kőszegi dominion gradually expanded into the region Baranya, both Conrad and James became a loyal supporter of Andrew III of Hungary.
Conrad and James were granted exemption from the jurisdiction of the ispán of Baranya County by Andrew III of Hungary, but in reality this did not prevail due to the anarchic conditions.
[8] Nevertheless, the Győrs' possessions were constantly harassed by the troops of Kemény, son of Lawrence and Philip Kórógyi, their powerful neighbors.
During the era of interregnum (1301–1310), he had various conflicts and lawsuits with his neighbors (e.g. Lawrence Tétény), the Pécsvárad Abbey and his brother-in-law Miske Rátót.
[4] Thus, it is plausible that James and his family swore loyalty to Charles I of Hungary, who aimed to abolish the oligarchic system after his third coronation in 1310.