There are probably reasons kanji pronounce differently as time and traditions have altered the language (kunyomi, onyomi, etc.). Just remember, just like they practice and practice it growing up, we learned English the hard way, too: over years. It’s how we got the point so we can say, “English is difficult. It can be understood through tough, thorough, thought, though.” So, expect years of practice
Also, it will be hard to learn if you aren’t continually using it in everyday life, or studying grammar. Ideally, you are studying verbs, conjugations, and usages with hiragana for a good time when kanji + furigana begin to seep its way into the sentences, starting with things like 私（わたし）so you can see both at the same time. It’s important if you want to learn the language, to not solely rely on kanji. Books and a native Japanese conversation partner will work wonders.
Good luck! And keep at it. 頑張って！