Hey @Termea,
Welcome to the forums!
What kind of Japanese usage are you hoping to get? for example - Being able to write or read emails / essays / business briefs, conversational (business vs. casual), teaching Japanese to other learners, etc? Depending on what is the most important part is, there may be some shortcuts for you.
Like you said, if you don’t use, it starts to go away, so the best plan of action is to never stop using it.
In my neighbourhood, the university and the community center (community learning) have meetups open to the public to practice different skills - some are for reading (they actually sit together and read through newspapers & magazines together), composition (a retired teacher volunteers his time to correct short essays on weekly topics), and there’s another that do conversation practice over sushi dinner once a month.
If you are somewhere remote, or someplace with not many other Japanese learners, there are practice groups that do online meetups via video chat.
I personally love reading, so I just find books that interest me and read them in two languages at the same time (like Harry Potter). 
Also, watching Japanese movies / tv can also be helpful, just to get used to the speed and tempo of the language again. Don’t worry about following along too closely, and turn on subtitles if it really drives you crazy.
I know it’s tough to get started again when you get the stupidest grammar stuff wrong after a long break, but I suggest focusing on what you do know and then keeping the Japanese part of your brain always running. Then the little mistakes are less important (and easier to recognise/correct) when the motor is already running.
Best of luck!