I always assumed that review ordering is random when you do fewer reviews than the total number of reviews available, but lately, I keep getting the feeling that WK must have tweaked the algorithm so that recently missed items come up a lot more often than chance.
For example in my last two review sessions just now, the proportion of leeches that came up for review was so high that there was only a 1 in 800 chance of that happening by chance, which seems pretty suspicious. And no, I don’t have any scripts installed.
It’s completely random, the only thing not random is that you can only have 10 active items at a time and new items only get put into that list if you complete one.
Probably a case of frequency bias if I could guess
The only thing that’s random is the order of items within a single review session. The items which appear during the session are based on SRS intervals and how frequently you’ve made mistakes in the past.
Yeah, obviously I know about SRS intervals and so on. I’ve been doing WK for years and already gone to level 60 and back. I was hoping that was clear when I said in the original post
I always assumed that review ordering is random when you do fewer reviews than the total number of reviews available
That doesn’t seem to add up. If you had 200 reviews and only did 100. If you happen to get all your leeches in the first 100, then the second 100 can’t have any of them.
Ok, here’s some specific numbers if it makes things easier for you:
I had 210 items that I designated “leeches” (note that this set was chosen before I did any reviews today), and 3407 reviews available.
In my first session today, I did 24 reviews and had 5 “leeches” come up. We can use a hypergeometric calculator to calculate the odds. With a population size of 3407, population successes of 210, and sample size of 24, the odds of getting 5+ are 0.013751326, or ~1.4%.
But hey, maybe that was just a fluke, so I did a second review session. This one wasn’t quite as lopsided, but it was still unlikely, as I got 6 out of 52. Since this was just half an hour later, nothing from the first review session would come back up yet (shortest interval is 4 hours), which makes things easy. Population size 3383, population successes 205, sample size 52 gives odds of 6+ at 0.091833235. (9.2%)
Combining the two gives 0.001262828 or about 1 in 792. I hope that helps.
The odds calculations are correct, but the thing with statistics is that a non-zero probability means that an event can occur. You can’t know for sure if your scenario is common until you’ve done a lot of experiments. To begin to get reliable data you would need to repeat the experiment at least 30 times.
Just out of curiousity, what if the algorithm had been tweaked to bring leeches toward the beginning of the queue. Would you consider that good or bad?
(I’ve been doing a little bit of leech research in my spare time, so I find this stuff interesting)
They’re actually not random, they follow the pattern of the Illuminati Bible Code. If you can reverse engineer the Code you can use it to pick winning lotto numbers…
