I’m not quite sure what you mean by ‘non-Japanese speaking context’. Do you just mean not getting any speaking practice?
In order to be properly conversational, you’re going to need speaking practice. When you’re in a conversation, you have to think up what you’re saying on the fly, so If you’re only used to remembering words in a review style context, you’re going to struggle both listening and comprehending what the person talking to you is saying, and trying to think of what you’re going to say next. There’s no way to really get around this though, without actually speaking to people in Japanese. Personally, I wouldn’t worry too much about having poor speaking skills to begin with. Japanese people don’t expect foreigners to be able to speak any Japanese, so being able to speak it, even badly, will be impressive to them.
One thing I do sometimes is, If i’m in the shower for example, I’ll try talking to myself in Japanese. Doing this should give you a good gauge of your abilities with being able to actually conversational produce sentences.
I’m going to stop editing this, I’ve edited it like 6 times now