Keep at it. At this early stage, your brain is still trying to figure out what you’ve gotten it into. At first, it will want to pick one thing to remember and simplify. It’s what brains do. They try to figure out a simple pattern first. One reading, one meaning. After a bit, when you’ve got more words, more kanji, your brain will adapt and start finding deeper truths that it can latch onto. Those will be wrong too, because kanji are perverse, but they’ll be right more often. That’s progress.
Others have said it gets easier the further you go. Fewer weird readings. I think this is because the simplest kanji (fewest radicals and strokes, so easier to recognize) are generally also core language concepts. And in Japanese, like in all languages, core concepts are the most likely to be irregular. Take counting works in English. We have “one”, “two”, “three”, but also “first”, “second”, “third”, and then there’s “single”, “double”, “triple”… Not to mention special purpose words like “pair”, "couple, “dozen”, “score”, “hexagon”… Ow!
Kanji adds a unique spin to this. Since the kanji gives you some meaning, you can often make a guess. Since kanji have common readings, you can make a guess. But neither guess is really certain. The kanji could have been chosen for their readings, but the meaning of the kanji are irrelevant (carrot, anyone?). Or the reading could be grafted on to the kanji later based upon the meaning, so the reading is totally unique. Nothing is really certain. They can be irregular, uncommon, or just perverse. Sometimes all three.
All that being said, there is a point, and it’s not that far in (I think I hit it around level 5 or so) where you start developing an intuition for it. You’ll just know, or at least suspect, that something funny is going to happen.
And you’ll be right. Cause it’s kanji.
And you’ll smile to yourself, and enter the right answer.
Good luck!