Maybe try “Laid Back Japanese” - they have a Japanese-only one as well as a combined English/Japanese one - the former would probably be more of what you’re looking for.
Look on YouTube - but some info about the podcasts is here:
Are you looking for podcasts aimed at Japanese learners? Or any native content?
For me personally listening to japanese streamers or youtubers filled the listening niche. Most 雑談 (free chatting) steams or non-gaming collabs feel like podcasts.
But to be fair there’re no training wheels with this type of content. I think you’d need to be at least intermediate level to benefit, otherwise there’s going to be too many unknowns: not only casual expressions, slang, but also normal grammar and vocabulary.