I think you may have a hard time if you don’t get a little more into N4 grammar at least. That’s all my level was, and I got through okay. But there’s a lot in N4 grammar that’s going to be all throughout the book.
Also, if you have any interest in manga, it might be a good idea to read Aria the Masterpiece with us first as a primer, then read 魔女の宅急便 right after. The manga will be much easier than the book, but it will still give you a lot of good practice with trying to understand real Japanese material. (For reference, I read one volume of よつばと before reading 魔女の宅急便.)
I also wanted to specifically respond to this. Of course you want to make sure you don’t get completely lost, but it’s okay if you don’t understand every sentence entirely. I learned this the hard way reading this book. At the beginning I tried to understand everything, but this just lead to frustration and burn out. Later on I found the right balance between analyzing a sentence and moving on to continue enjoying the book.
Other than that, make good use of the vocab sheet and the existing questions and answers in the 魔女の宅急便 chapter threads. And always feel free to ask new questions!