% correct on review summary screen calculation change?

Somehow, my accuracy according to the reviews screen is sky high (90+%), but my actual session accuracy is not that high (generally 75-85%).

I do use Flaming Durtles (android app) quite a bit, only use the regular web interface on weekends.

Did something change? It used to be more reflective of the total cumulative reviews.

I think the percentages during reviews and at the end of reviews (on the summary screen) are calculated differently:

During reviews, the little percentage in the upper right corner determines the percentage of correct “review events”, let’s call them. Each Kanji or Vocab item has two “review events”: reading and meaning, only Radicals have one. So if you review 10 items (let’s assume no Radicals for simplicity) then that’s 10 items but 20 “review events”. So if you fail e.g. one reading, that means 1/20 = 5% failure rate, i.e. 95% correctness during the review session.
In the summary screen, items are what counts, i.e. in our above example, you failed one item, that means 1/10 = 10% failure rate, i.e. only 90% correctness.

AFAIK this has always been calculated like that, but I have no idea why tbh :slight_smile:

8 Likes

As pointed out by @NicoleRauch, these two numbers are calculated differently, but I really don’t understand why. Will this be changed any time soon?

I don’t know if it’s on the WK team’s to-do list, but there is at least one user script that addresses the issue, for anyone interested. ^^

The script seems to be from 2013, and the link to userscripts.org does not work for me any more, nor could I find it on Greasyfork… which is a shame, because I got really curious how the rating algorithm was implemented.

Because thinking about it again, I suspect there is no reliable way to produce good results mid-way through a review, given WaniKani’s review system where all “review events” of all items are mixed.

Here’s why: If you again review 10 items, and for some reason you first get to review the 10 meanings, and you get them all correct, what is it supposed to display? It cannot display your true percentage because that is not yet known. So, should it display nothing for the first half of the review time? Or 100% because that’s what you can potentially still achieve? Or 0% because you could still fail all of the reading reviews? Either of those would feel weird to me.
If, on the other hand, you get 5 of the meanings right and 5 wrong, 5 items are wrong for sure. You can get at most 50%. Should you see 50% after reviewing those 10 meanings (because that’s the maximum possible value) or 0% (because that’s the minimum possible value)? Again, I think that’s weirder than what WK has currently in place…

So i guess we need to live with what we have :upside_down_face:

Nope, that is not what I am talking about.

If you use an offline apps, the review percentage accumulates until you log back in and do reviews live on the website interface. This used to show a fairly accurate summation of the cumulative review percentages. So if I ran at 100% for 20 reviews, the 80% for 100 reviews, etc, it would show the percentage for the total reviews that I did.

Now, it appears to be taking only the last passed review and throwing away all of the previous reviews on that item, which is artificially inflating the cumulative review percentage for the week.

Oh, I see! At least for the web interface, it has always more-or-less been like that (I’ve sometimes seen aggregations of results, but only over a very short time span, not over a whole week). What offline apps do is of course their own thing; maybe they send all accumulated reviews in one go so that WK thinks it has just been one review? No idea tbh…

Yeah, I realized this could be an FD issue - will ask the developer!

I noticed something similar too. I had like 85% correct when I hit “Reviews” in the browser (the screen before you start reviewing session). I was doing reviews with Flaming Durtles for a week or so only, but never got that high correct % there. I assume they just take all vocab/kanji you reviewed in that week and show you if it was either wrong or right the last time you reviewed it. So, if you reviewed a kanji 6 times, failed it 5 times (and thus should bring down your %) and got it right the 6th, it will show as correct and therefore present a completely arbitrary % on that pre-review session screen. I can’t remember if it was any different before, though.

Edit: Oops, that’s what @atomicalex already described 2 posts above. It did feel like it showed a more precise percentage before, though. I just recently noticed that something might be off, but not sure at all.

1 Like

For context, this is how Flaming Durtles calculates the percentages at the end of each session:

First of all, only the items actually quizzed in the session you are finishing are counted. FD does not look at any past review results.

Then, FD counts the number of items in the session for which no mistakes were made. That is, both reading and meaning questions were answered correctly on the first try.

Then, that number is divided by the total number of items in the session, and shown as the “Overall percentage correct”.

And finally, the “Overall percentage incorrect” is simply 100 minus the “Overall percentage correct”.

So if you had a session with 10 items, and you made one meaning mistake, your overall percentage correct is 90%. If you had a session with 10 items, and you made three meaning mistakes and two reading mistakes on a single item (and the other 9 items were correctly answered every time), then the overall percentage correct is still 90%. If you make one meaning mistake and one reading mistake for two different items, your percentage correct will be 80%.

This is how it’s been calculated since the first release of FD, and I don’t think there are any bugs in that piece of code. I don’t remember why I’m calculating the percentage exactly like that, but it must have been to duplicate the calculation on the site itself. Or maybe a popular userscript.

3 Likes

Yeah, I don’t think it’s due to FD. My guess is that WK just disregards all previous reviews since the last time WK was used for reviewing in the browser and only counts and presents the very last review result of each kanji and vocab. I guess they might have fixed the percentage so that it’s in line with the “answered correctly/incorrectly” list below. If they changed anything at all, that is. lol

@Kappa420 That is exactly what I think is happening. Which makes the cumulative review screen less helpful to me. :frowning:

@ejplugge thank you. I had trouble with the idea FD could be the issue. Thank you for commenting!

1 Like