I get what you’re going through, up to a point. I’m not sure I have any direct recommendations for you, but let me tell you about my experience and maybe it’ll help.
For some background, I’ve been studying Japanese for about 4.5 years. I’ve read 7 books in Japanese and dozens of volumes of manga. I’ve never played any games in Japanese, nor have I seriously tried watching anime without English subtitles.
First, it’s okay to do things in English. It’s not giving up. I recently played Steins;Gate and Steins;Gate 0 (my first VNs), and I played them in English. Because I wanted to relax and have a good time, rather than struggle reading this complex sci-fi narrative in Japanese. Also, the main reason I’ve never seriously attempted to watch anime with Japanese subtitles or without subtitles is because anime is one of my primary means of unwinding. I don’t want to have to think too hard. Should I try to watch more without English subtitles? Probably. But I don’t really care because I’m enjoying myself.
Now onto reading. I can certainly relate to the “I understand all the grammar and words, why can’t I make sense of this” thing. I read a light novel late last year, and I constantly felt lost. I’d read the same sentence over and over and just not be able to make sense of it. It was so bad that I had to read the book in English in parallel in order to follow the story (I’d read each chapter in English after reading it in Japanese). However! I don’t relate to the thought that Japanese is never fun. I sometimes find a book where I can read pages at a time without feeling lost. Yes, I have to look up a lot of words, but if I can read several pages without feeling lost I consider that a win. Additionally, I can truly say that I read manga for fun and not for studying now. Again, I still have to look up words, and yes I sometimes get confused or miss the punchline. But overall, it’s an enjoyable experience and something I look forward to.
So do I have any real suggestions? I suppose it sounds like maybe you’re trying to read the wrong stuff. Maybe you’ve jumped too far into the deep end and haven’t given yourself a chance to ease into reading native material. I can’t really say. It sounds like you need to find some interesting but easier manga/books/VNs to motivate yourself while still having a good time. And as always, I recommend joining a book club, which is a great way to read something while being able to ask for help and discuss the story with others.