You know who seems to be distressed all the time? Little 子s. 子s are always crying about something. Always. And when adults are distressed they can look like 子s too.
So the reading for 子 is こ. But the reading for 困る is こまる. Not sure how the above mnemonic reflects that?