Indeed in any conversation you’re most likely bound to be listening if you aren’t as fluent as the other person, so seems like something to take into consideration.
I would suggest start with real material, choose something somewhat easy of course. Rewatch some shows you know by heart only this time in japanese, no subs, no nothing… just your two ears and atention. Look for dubbed movies too… basically anything where the plot it’s not an issue.
there’re some genres that are more suitable to provide clear audio in my experience… slice of life in case of anime… and well basically dramas have turned to be better immersion in my case fro the same reason (but it’s probably more related to likings and such). Also Netflix originals in japanese have an audiotrack for visual impaired people… watch the shows with that track… it provides a super clear description of everything happening… so If you miss dialogues or the nuance of a given situation, it can clarify a lot. besides the narration voice it’s super clear.
I’m gonna make a suggestion somewhat uncommon, but get noise cancelling headphones if possible. I did this at the start of the year, and I realized the sounds are way clearer!! As if they were made for language learners … seriously… If you’re care about hearing… well those are a great tool.
Overall, is like everything. start… repeat… do it again … make a routine out of it.
If you are compfortable with not understanding everything… then passive immersion (like playing and mp3 player with the audio of your shows or audiobook, or similar) also can be a great opportunity to fill some iddle moments… commuting and supermarket time …
Lastly I will mention audiobooks, 'cause is something I’ve been using, and it’s turning into a great adition to my current reading.
Specially if you are reading something… of have read in the past. Look for the audiobook… make that your passive immersion. It’s great quality narration most of the time. Audible has a free trial, actually you can look for some of the books in the american Audible site (they give you 2 ) in japanese, though with less options, AND then also in the Japanese Audible site… heck those are 3 audiobooks at zero cost.
Anyway… that’s all I can come up for now. But hey, there’re lots and lots of way to improve your listening. just pick one and make it into an habit.