Nothing wrong with slowing down. I’m doing 12 lessons a day.
With the default ordering, that’s actually not that surprising, because it will give you all current level kanji before any current level vocab. To avoid this, but still level up consistently, I use the Lesson Filter script. I do 3 kanji a day and up to 9 vocab a day (if they are available). This lets me level up consistently, but not have this backlog of vocab that you’re concerned about.