Science Behind WK Method

I know this sounds inflammatory and confrontational, but it is not intended that way at all.

I just read a disturbing thread regarding the potential effects of the stress caused by the WK workload (suicidal thoughts were mentioned), and it made me wonder how trustworthy the methodology here actually is. I looked at the Tofugu staff board and saw links to articles, but no educational credentials or CV, etc.

Where did this curriculum come from, and what are the qualifications of the people who created it? Again, this is not a criticism or accusation - I am genuinely curious, and think it would be helpful for everyone to have this information to put things into perspective going forward. If this has been discussed before, please point me in the right direction. I wasnā€™t getting much from searching.

1 Like

There are two core features of WaniKani:

  1. Mnemonics
  2. Spaced-repetition

Itā€™s the latter than would be relevant to this thread.

The spaced-repetition is based on a system developed by Sebastian Leitner. I donā€™t know his background, but he wrote a book called So lernt man lernen (How to learn to learn), which Wikipedia refers to as ā€œa practical manual on the psychology of learningā€. Iā€™ve not read the book (if thereā€™s an English translation), so I donā€™t know what is may cite.

With the wide use of spaced-repetition systems, typically based on the Leitner system, I think thereā€™s a large amount of evidence about the general usefulness (positives) of these systems. Iā€™ve never looked into any studies on it, whether older or recent studies.

It would be interesting to see studies on the negatives, but my own personal expectation is that things like suicidal thoughts would point to other issues, that any kind of stress can result in. (But Iā€™m not a psychologist, and have never studied anything in this area.)

If youā€™re curious enough, I recommend looking up studies on spaced repetition and the Leitner system, as well as the aforementioned book.

You may already be aware about SRS, etc. I wrote this reply to be fairly general to the system used by WaniKani.

17 Likes

To be honest, Iā€™m not entirely sure how this is really a thing? Maybe Iā€™m a little ignorant here, but Wanikani is set at the pace of the user, right? So, itā€™s really up to the user and how they pace themselves. They should be aware of how much work they can take at a time based on their past experiences. Also, Iā€™ve seen people argue that WK is way to slow at teaching kanji.

3 Likes

I am indeed aware of the idea behind SRS itself, and certainly wasnā€™t trying to draw a line between it and suicidal thoughts (clearly something else is going there - but it got me thinking about the demands of WK in general), but WaniKani has its own implementation, pacing, vocab load, curation, etc., so itā€™s not just a question of SRS. Iā€™m more curious about the total authorship of the curriculum and why it is to be trusted (again, I know that SOUNDS confrontational, but it is really just a question of basic vetting).

2 Likes

Personally I am not worried about the effect of Wanikani. Some people may have psychological issues and Wanikani may act as a trigger, but this doesnā€™t speak to a problem with Wanikani. It is just that some people have issues that must be addressed. These issues would be present even without Wanikani.

12 Likes

My expectation would be that thereā€™s no science behind what WaniKani settled on for its implementation of time spans between reviews. Does anyone who was here in the beginning know if there are an era of tweaking the values early on?

For me personally, the two main reasons itā€™s to be trusted are:

  1. Many people have reached level 60.
  2. Itā€™s working for me, so long as I go at a pace I can handle.
7 Likes

Oh definitely. I alluded to it above, that I am not certainly not accusing WK itself of pushing people into an unhealthy mental state. But I am questioning (in good faith, I promise) how the certain amount of trust required to follow a system over a long term like this is justified by science, research, qualifications, etc.

1 Like

Also, for me, I think it can be trusted as much as any online language learning resource.

  1. People use it, so the content must be mostly accurate/helpful. Otherwise it wouldnā€™t be successful or profitable and people wouldnā€™t recommend it.

Itā€™s the trust of the community. I have no idea what credentials the people who write the language learning resources I use hold. But if they werenā€™t mostly accurate and if they didnā€™t work, they wouldnā€™t be used and recommended. Itā€™s vetted by the users.

3 Likes

Itā€™s worked well for me as well in terms of reading, but i reach a point where I needed more flexibility to make it practical. The rigid adherence to the program without any customizations does kind of imply that there is a reason, doesnā€™t it? I guess that is kind of a corollary to the original question.

1 Like

Every SRS system I use (Wanikani, Bunpro, Anki) has its own system for SRS intervals. This leads me to believe there is not that much scientific foundations behind it because otherwise everybody would use the same intervals. @MegaZeroX has made an investigation on a similar but not quite the same topic. Here are their findings.

8 Likes

Not entirely. WaniKani is one of the systems that gamifies the whole thing. And that means that the pace is partly being set by the game. In this case, the leveling system. Gamification can be fun and effective for some people, but it can also cause problems when users let the game run away with them. And not everybody can properly handle that and keep control in their own hands. For some, learning kanji becomes secondary, and speed-leveling becomes the primary goal, much more important than actually learning. ā€œI have to do all of my lessons right NOW, because this is a GAME and I have to WIN, d*** it!ā€.

Yes, Iā€™m exaggerating, but thatā€™s the idea. Letting the game take over can cause a lot of stress, and that can feed into burnout and other mental health issues.

5 Likes

I think you are lumping two distinct questions together that are wholly separate:

  1. Is WK effective? Does it fulfil its purpose of aiding the user in learning kanji efficiently?
  2. Does WK have negative impacts on its users?

The first one is what most people are answering here, and there appears to be ample scientific evidence for the claim that mnemonic-based memorization and spaced repetion are effective tools for acquiring factual knowledge. WK is more rigid than other systems because itā€™s fully productized. It makes executive choices to lift the burden of making those choices from its users, in the same way that, for instance, Apple makes executive choices about how iPhone apps have to look and function to ensure a consistent user experience.

What youā€™re actually looking for, though, is a discussion of the second question.

(Disclaimer: Iā€™m not a doctor or psychologist.)

As ejplugge said, WK is employing several methods that are commonly lumped together as ā€œgamificationā€. Putting a level number next to your username makes you want that number to go up. Most actions you take give some kind of reward, from guruā€™ing an item to unlocking lessons to leveling up. These strategies are widely used to form habits. Often this is done with nefarious or at least dubious intent, e.g. social media apps are specifically engineered for frequent randomized rewards to keep the user from engaging with the app more (which leads to more ads viewed inbetween posts and thus more money for the social media company); a fact which is not sufficiently disclosed to users. (Remember the outcry when it was revealed that Facebook turned its users into guinea pigs for a study that proved that a different feed algorithm made people angrier?)

However, habit-forming techniques are a tool that can also be used for good. Suppose that someone is a couch potato, but they want to get into the habit of exercising. If they can find a method that manipulates them into exercising, thatā€™s a good thing in their book.

So Iā€™m not condemning WK for using habit-forming gamification techniques. If they didnā€™t do that, Iā€™d probably have given up on learning Japanese already, so their gamification is entirely in my interest. But when a habit-forming thing meets the wrong person with the wrong state of mind (for whatever reason), thereā€™s a risk of unhealthy addictive behavior.

If I were Tofugu, I would try detecting usage patterns that hint at an unhealthy use of their system. If someone does reviews ten times each day for a long time, that looks like obsessive behavior and the system should try engaging with the user to advise them to give themselves some time to breathe. This is just an example. It would be a great idea to engage with medical professionals to gain insights into usersā€™ mental health and discuss which actions to take.

5 Likes

I think it they may be similar, as it would make money because of premium users, so impact and usage could relate in ways.

Except when you do not use a set schedule for your reviewing, but do them whenever you can squeeze them into your time, thus breaking reviewing up into many small segments.

Thatā€™s how Iā€™ve been doing it all along. And I feel fine doing it that way. I do not want to do 150 reviews in one go, then 100 more later and so on, but rather 20-30 at a time.

There is no way to generalize the use of the WK-system in such a way I think.

2 Likes

You put thing in brain, it stays in brain. Next time you put it in brain space is longer. If it doesnā€™t stay in brain try again.

1 Like

Chukharev-Hudilainen & Klepikova (2016) conducted a small experimental study that was able to show nearly threefold improvement in the retention of vocabulary for EFL students through a spaced repetition system, but they also pointed to a lack of ā€œformal models of word acquisition stages in vocabulary tutoring algorithmsā€ (p. 351).

Seibert Hanson & Brown (2020) tested the effectiveness of Anki for beginning college learners of Spanish. While ā€œthe results showed a positive relationship between number of days studying with Anki and level attained of Spanish even while controlling for baseline abilities and for motivation, self-efficacy, and epistemological beliefs at both the beginning and end of the semesterā€ (p. 147), they also reported several limitations related specifically to the use of Anki. Some learners ā€œdisliked studying words from chapters they had already been tested onā€ (despite the fact that thatā€™s the point of SRS) and noted that ā€œthe applicationā€™s interface was simple and not engagingā€ (p. 149), which led to learners using it inconsistently.

The authors note that these results point to an extrinsic rather than intrinsic motivation (which happens a lot in a college language context) so that is likely to be different for most Wanikani users.

Chukharev-Hudilainen, E., & Klepikova, T. A. (2016). The effectiveness of computer-based spaced repetition in foreign language vocabulary instruction: a double-blind study. CALICO Journal, 33 (3), 334ā€“354. https://www.jstor.org/stable/90014364

Seibert Hanson, A. E., & Brown, C. M. (2020). Enhancing L2 learning through a mobile assisted spaced-repetition tool: an effective but bitter pill? Computer Assisted Language Learning, 33 (1), 133ā€“155. https://doi.org/10.1080/09588221.2018.1552975

4 Likes

Could you elaborate?

Itā€™s a turnkey product. In other words, the user doesnā€™t have to worry about setting anything up - they just show up and go. You follow a set and familiar routine from start to finish and donā€™t have to think too much about it.

If a user wants their own bespoke solution they can certainly do so with a platform like Anki, but then thereā€™s more time invested in collecting the source data, figuring out how they want to order it, what SRS schedule they want to follow, etc.

There are three things here:

  1. The åøøē”Øę¼¢å­— list which comes from Japanā€™s Ministry of Education
  2. Mnemonic-based learning, which goes back centuries
  3. Spaced repetition learning, which likewise has been around a while

Itā€™s not rocket science. The WK platform just puts together stuff thatā€™s already been well-established.

3 Likes

I donā€™t think that WK is responsible for or causing mental health issues in its users. If someone feels overwhelmed by the workload then there are options such as slowing down, taking a break, or stopping entirely.

It might exacerbate a pre-existing mental health condition but thereā€™s no way itā€™s caused by it and if WK is causing excess stress over and beyond minor annoyance and irritation then a person should seek real life medical assistance.

I think itā€™s unfair to expect a company to snoop into a personā€™s medical health when providing a service.

Iā€™m curious as to why you think the mental health of users is Tofuguā€™s responsibility or why they have any right to that data and information.

In the EU data held by companies has to be for a legitimate purpose and as Tofugu holds data on EU citizens they have to control and process data in accordance with EU law. I donā€™t see how thereā€™s any legal basis for them to collect data on usersā€™ mental health nor do I think itā€™s appropriate.

7 Likes

Where did this curriculum come from, and what are the qualifications of the people who created it?

In truth, Wanikani doesnā€™t have a curriculum per se. Iā€™m not trying to downplay how nice Wanikani is, but in truth itā€™s really just a very nice SRS flash card system that requires you to type in the answers. Unlike a structured curriculum, it doesnā€™t really make any guarantees of what youā€™ll be able to do with your Japanese language skills after burning all of the content they have in their database. Itā€™s a really nice learning aid made by developers who are passionate about the Japanese language and Japanese culture. It is not a replacement for any structured curriculum.

Itā€™s worked well for me as well in terms of reading, but i reach a point where I needed more flexibility to make it practical.

You need to develop a personalized learning program (strategy, not software) for your needs. This is not a unique problem youā€™re facing, itā€™s an eventuality in every language learnerā€™s journey.

When devising a curriculum, it is good practice to build or identify exercises that develop Transfer Appropriate Processing, which means that the exercise should mimic realistic situations as closely as practical. Wanikani only ā€œtestsā€ your ability to recognize the reading and meaning of isolated kanji and vocab. It does not ensure you know how to use them (it provides example sentences, but it does not test you on them). To go further, you need exercises that help you develop an understanding of the nuance of these words. Reading books helps in general, but you also need to learn practical usage of everyday vocabulary as well. Do you know how to read a Japanese address? An advertisement? How to fill out a form / application? What are the common words used for navigating a website? What vocab and grammar structures do you need for communicating in your chosen career / profession?

To decide how much you should use Wanikani, you first need to define your objectives and identify which tools & coursework supports those objectives best.

5 Likes

Iā€™m not saying Tofugu should collect health data. Iā€™m saying that there may be patterns which may hint at an unhealthy use of the application. In these cases, they could send notifications advising users in neutral language that unhealthy use is in fact unhealthy. I donā€™t think itā€™s that different from the automated engagement they do anyway after level-up, or when you havenā€™t checked in to do your reviews in a while.

For instance, something like ā€œWe see that you do a lot of reviews each day. If thatā€™s what fits best with your daily routine, thatā€™s great. But please remember to not let WK reviews take over your entire freetime.ā€ You see that Iā€™m not that good at that kind of writing, but I hope the idea comes across.

(Also, since data protection came up. Disclosure: I occassionally do press relations for a EU-based NGO which among other things acts as a privacy watchdog.)