Kan-on should not be confused with tō-on (唐音, Tang sound), which were later phonetic loans.
Kan-on are on'yomi (音読み) based on the central Chang'an pronunciation of Middle Chinese.
[1] The syllable Kan is a reading of Middle Chinese: 漢 (xanH) as per Japanese phonology, referring to the Han dynasty, which had Chang'an as its capital city.
[1] Furthermore, Kan (漢) has also become a description for all things Chinese, e.g., kanji ('Chinese characters').
A few dictionaries go as far as to discard attested kan-on in favour of more systematic pronunciations.