I’d suggest hitting up the book clubs first, since having people to read along with and ask questions can help:
There are several manga listed in there already, and people kind enough to make vocab lists too! Definitely take advantage of it if you can 
If you can manage to squeeze in some grammar studies, like others mentioned, it will make things easier. Even just getting down N5 grammar first will give you a huge boost. (N4 is where the difference really starts when it comes to reading though.) Doesn’t hurt to try, and if it doesn’t fully work out, jump back to your old method. 
Best of luck!