Quality of Response Measurement

I have been using WaniKani for a few months and am starting to research the topic of SRS on the side. After some research, I’ve found that WaniKani and several other SRS apps these days have their scheduling based on the SM2 algorithm. I was wondering if anyone is privy to how WaniKani rates the quality of response to ensure I am progressing through my levels as efficiently as possible.

Some thoughts I’ve had on how they could rate between 0-5 as the SM2 algorithm relies on.

  • Time based - less than 10 seconds, less than 30 seconds, etc.
  • Partially correct - Answers that have some of the correct hiragana might get partial credit and award you a 2 instead of a 0 for failing

Thanks in advance for tips on helping me better my leveling and/or educating me on the SRS scheduling that WaniKani employs.

Even Koichi has admitted that the WK SRS intervals may not be optimal/perfect. On the other hand, I’m not convinced that we have to find the “perfect” SRS intervals to be able to learn. Yes, some research went into the SM2 algorithm and it’s still being improved all the time. But using SM2 as the algorithm basis for WK SRS is good enough.

2 Likes

Maybe the perfect interval depends on the person as well

2 Likes

I’d say that the WK intervals are completely fine, as they allow you to build a very nice schedule with your lessons and reviews. The only thing missing from WK is the lack of SRS really (i.e. the ability to do more quizzes of the lessons learned). I believe some binge of recent learned items together with SRS really solidifies the whole thing much better.

To understand what I’m talking about when I mentioned WK’s schedule, please read chapter 5 of my guide (if you haven’t already):

2 Likes

If you’re asking about how WaniKani works, I don’t think they judge answers beyond right/wrong. This article in the FAQ has some more detail about how WaniKani calculates SRS level.

4 Likes

@fuzzytipsy Thanks for the link and your response. I’d wager you’re probably right, and if so, they have their own formulas they likely use instead of the exact SM2 implementation. Unless if you get it right your quality is 5 and if you’re wrong the quality is 0. That is totally possible too methinks.

Thanks for the reply @jprspereira. When you mentioned the part I qouted above I was a little confused. WK is built around the concept of SRS. The thing missing you’ve mentioned is more of a cramming feature that other apps like Anki and Mnemosyne have.

So for you the ideal WK would be the SRS with the addition of a feature that would, for example, take the last items you’ve learned in the previous week and construct a quiz for you. This would be solely independent of the SRS, as in it wouldn’t affect your levels or scheduling.

Does that sound about right?

2 Likes

Wanikani has the quiz after lessons :b

Yup, that. Although I would only use it with leeches and learned items from last session. I’ve recommended some people to review the items they just learned 5 to 15mins after learning them and I’ve only heard of positive results like “I’d get x% accuracy and now I get them all right at Apprentice 1”.

Nevertheless, there’s a script called Self-Study that does what I want. I just wish there was more push for something like this native to the platform, as I believe it would improve people’s learning significantly.

It probably does, but that’s tough to solve. If you leave the configuration of the intervals to the user, everyone will just pick way too short ones and just overwhelm themselves. I think the ideal way would be to somehow try varied intervals during the first few levels and measuring how well the user does with those, then setting the most successful intervals for future levels.

1 Like

That would be awesome actually!!

The problem is of course whether the user can find the time to keep up with the proposed intervals so the first testing is accurate. In theory this would be so awesome though

1 Like

Well the SM2 algorithm attempts to find ‘better’ (not perfect because it is far from that) intervals by adjusting the interval each rep with something called an easy factor. This will give you a much more dynamic routine as you progress based off the quality of your response.

However, the more I research WK it seems they aren’t using SM2. The reason that I say that is there is a strict spec that the author put out on being SM2 compliant. It is located here:

https://www.supermemo.com/english/ol/sm2.htm

Now I don’t have access to WK source code. I’d love to see it if it was open source as the JS served to the frontend is minified and ugly as all hell to parse… but it seems WK does not follow those principles.

If you look at @jprspereira extremely well written guide, there is a slew of info on the scheduling including an image with the intervals tied to the SRS levels. These intervals are all static! 1 week, 2 week, 1 month… you get one wrong you pop back a level or two based on a formula of your SRS level. These intervals are static and never changing. There also isn’t the concept of quality of response either. Only right or wrong. Not 0-5 as I’ve asked in my OP.

I’m definitely not banging on WK, the platform is genius. I’m just the type who loves to dig to the core of everything I love and it would seem those who say it is based off SM2 are stretching it. Calling Mnenosyn and Anki based off SM2 is for sure , but WK is almost more like the Leitner System.

Sorry if this was a little all over the place. Just woke up and am on mobile.

3 Likes

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.