I passed the N4 this July and finished the Genki Books shortly before. In terms of my experience, the gap betweeen where you’re at and where colloquial japanese is much of the time will be sufficient for you to express yourself in Japanese, but not often to fully understand what people are saying to you if they don’t have any knowledge of english (ironically, this is a strong indicator of whether or not they are even aware what Japanese is “simple” or “beginners” Japanese vs. what isn’t). This sort of problem is twice or thrice as hard with elderly folks like おばちゃん who may have an accent, speak a different dialect, or more often than not, both.
That said, communication is about more than simply transmitting ideas– in japan especially, it can be about reinforcing social bonds and showing regard for people on a human level. And with that in mind, you should be able to engage the おばちゃん on that human level with what Japanese you have when it is possible, and (as has been the case for me) with generous gesticulations and pantomiming when your language knowledge doesn’t cut it.