I can only speak to my experience, but I used Genki I & II, which puts you at about an N4 level. Then I used An Integrated Approach to Intermediate Japanese, but really started reading a lot of NHK Easy, which is usually pretty straightforward after the teens/twenties of WK. The subreddit for NHK Easy automatically hides the furigana, and there’s also a script if you prefer to view it on the NHK Easy site, which helps with kanji recognition. As I got to a higher level, I relied more on Imabi, which is not very pretty to look at but is by far the best grammar resource on the internet for higher-level grammar points.
At that point, I started playing through old RPGs I’d already beaten in English, and with a general sense of the story, translating the dialogue wasn’t too much trouble. I played FF7-FF9, but Doublevil swears by Atelier Ayesha, which has the benefit of having voiceover you can repeat as many times as you like as well as text. Some people prefer children’s books or graded readers as well. Eventually, I set up news alerts for native Japanese stories around my area of professional work and started reading gaming news in Japanese as well.
My personal opinion is that you really need a grammar text for the foundation that gets you to N4, but after that you should just start reading native material and looking up vocab/grammar as you go.