Does anyone feel like they're not memorizing the kanji but rather how they look?

That’s mostly good though. You don’t want to have to break down every kanji all the time to read anything. The main things breaking down kanji is useful for is writing, differentiating similar looking kanji, and sometimes to help with meaning/reading (since kanji generally have phonetic and semantic components). For more about phonetic components, see [Userscript] Keisei 形声 Semantic-Phonetic Composition.

For that I’d recommend using Jitai (字体): The font randomizer that fits. This script will change fronts during reviews at random. You can install your own fonts and add them to the script if you want even more variation.