Yes, it’s easy to associate one idiom in English with another one that’s used in similar situations. I’m often left scratching my head wondering if my added synonyms are correct or a stretch.
In this case, I’d concur with others that “Never mind” (Nevermind is the name of an album by Nirvana) isn’t the same as 残念 which is really used more to mean “that’s a shame” or “what a pity” or in less grave situations, “too bad!” In more polite situations, the speaker would say, 残念ですね.