What’s helped for me with 人数 is, instead of the Sue I usually use for 数 mnemonics, I started using Zeus instead. So in this case, Zeus is counting people, his worshipers. Sue has no reason to count people. Zeus does, because he gains power from the amount of people worshiping him.
I dunno if this will help, but it turned from one of my bigger leeches to one of my stronger mnemonics.
My worst leech is apparently 八日, and I believe it. I can never remember if it’s よう or よっ and I always mess it up at the worst time.