Hahaha. Skipping around sounds like what happened when I had French class in school. Teachers never followed the textbook strictly (though they usually stayed within the same chapter for each lesson).
I personally only really bother with the texts, dialogues and grammar points. I also read cultural notes where possible. Still, there are some things I just glance through for general knowledge without expecting to remember them, because I think it’s impossible to do so without context/practice.
Separately, I gave myself a little kick today and started chapter 8. Culturally quite interesting (as usual), but since I’m more of an academic person (as much as I love reading fantasy books) and I want to read newspapers, I hope to find a chapter on politics soon. It seems that only comes later though. Learning about literature might be nice too, though I have no clue if I would enjoy Japanese classics. (I read about 吾輩（わがはい）は猫である, in which the author writes from the perspective of a cat observing humans. I didn’t know what to think, though the premise seemed interesting.)