Well, yes, if you read it without any help, Yotsuba is difficult. But there’s loads of support! There are vocab lists for volumes one and two, and the great Yotsuba reading club here on the WK forums. We’ve read through volumes one and two and are now on volume three.
I don’t even have N5 grammar, but reading よつばと！with all the good people here has been an amazing experience, and a billion times more interesting (I am sure) than reading a kid’s book full of “I have three apples”.