That’s fairly common =D
I did this with the first Japanese manga I was serious about reading all the way through, and it helped me a lot. Not only looking up words, but also any grammar I didn’t know. I learned a lot by the end of the volume.
Prior to that, any attempts I made with other Japanese manga I ended up making little to no progress. It’s easy to become discouraged or burn out. The important thing for me was to put myself on a schedule. Every day, I had to make a certain set amount of progress, no many how long it took. (It helps that my schedule allowed this.)
I tried to become as familiar with the grammar and meaning of all the dialogue I encountered each day, but I also allowed myself to skip over anything that was taking too long without making progress. Of course you can always ask questions on the chapter-appropriate threads here, which is a lot nicer than skipping over confusing lines.
Specific to しろくまカフェ, don’t worry too much about learning all the words in the puns (beyond finding out what they mean). They’ll often be uncommon (or rare, obscure) words that you won’t need to know for the entire rest of the book. There’s no harm in learning them, but I’d prioritize all words other than the puns first.