Is Refold actually going to make me fluent?

Japanese Language Questions
202

FWIW, the US Foreign Language Institute I linked above defines their target fluency as

Speaking 3 (General Professional Proficiency) Able to speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations in practical, social and professional topics. Nevertheless, the individual’s limitations generally restrict the professional contexts of language use to matters of shared knowledge and/or international convention. Discourse is cohesive. The individual uses the language acceptably, but with some noticeable imperfections; yet, errors virtually never interfere with understanding and rarely disturb the native speaker. The individual can effectively combine structure and vocabulary to convey his/her meaning accurately. The individual speaks readily and fills pauses suitably. In face-to-face conversation with natives speaking the standard dialect at a normal rate of speech, comprehension is quite complete. Although cultural references, proverbs and the implications of nuances and idiom may not be fully understood, the individual can easily repair the conversation. Pronunciation may be obviously foreign. Individual sounds are accurate: but stress, intonation and pitch control may be faulty. Examples: Can typically discuss particular interests and special fields of competence with reasonable ease. Can use the language as part of normal professional duties such as answering objections, clarifying points, justifying decisions, understanding the essence of challenges, stating and defending policy, conducting meetings, delivering briefings, or other extended and elaborate informative monologues. Can reliably elicit information and informed opinion from native speakers. Structural inaccuracy is rarely the major cause of misunderstanding. Use of structural devices is flexible and elaborate. Without searching for words or phrases, the individual uses the language clearly and relatively naturally to elaborate concepts freely and make ideas easily understandable to native speakers. Errors occur in low-frequency and highly complex structures. (Has been coded S-3 in some nonautomated applications.)
ILR

And similar for reading.

1 Like
203

Oh huh I didn’t realize that

204

Here’s at least one meta-analysis on the subject: English Programs for Internationals - English Programs for Internationals | University of South Carolina

Another study by Truscott of his own: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-1770.1996.tb01238.x

Anecdotally, I like to think how a lot of English learners have trouble with the third singular person, although they are taught it very early on in their studies and corrected regularly. Still, they continue to make the same mistake.

2 Likes
205

To chip in on the conversation on learning grammar, I don’t think this concept of it not being necessary to learn grammar is wrong. Even natives as children are taught the fundamentals of grammar as a child whether the parents realize it or not. Parents start by using simple sentences and correct their children when they use grammar incorrectly. Children are slowly introduced to grammar but you do not get this type of interaction from solely learning through immersion as the content you watch is not being catered for your personal level of vocab and grammar comprehension and you have no one that is correcting things you have misunderstood.

2 Likes
206

I’m not familiar enough with this particular problem to fully appreciate it, but I feel that being corrected when I made French conjugation errors definitely helped me become aware of a need to improve. Whether or not subsequent improvements in conjugation accuracy were due to the correction or due to exposure/habituation to examples of good language, I do not know.

Anyway, I spent a really long time trying to see if any research has been done on the other side i.e. has anyone tried to prove that correction helps? As far as Truscott’s review is concerned, something I disliked about the first sections of the literature review is that the results quoted didn’t touch on the sort of correction used. The middle portion of the review was more detailed, and I have to say that the results were surprising. It seems that this debate has been raging for quite a while though, and it seems the main academics opposed to correction are Truscott and Krashen. The fact that the debate has been running for so long seems to suggest that nothing conclusive can be said just yet, and that it’s not likely anything will be decided any time soon.

In any case, I eventually came across this 2020 literature review that cited another study as having shown that correction is helpful. I haven’t been able to see the full text because it seems it’s a paid study, but I guess it helps to know that it exists, and that the experiment apparently contained a control group. I don’t know how rigorous the experiment was beyond that: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9922.2011.00674.x

I guess the real question is whether there is any form of correction that works or is helpful. I personally believe that it’s possible for correction to be helpful if a student understands why something is wrong or how it changes what the sentence means, which can be something a grammatical error does; or if the correction provided helps the student create a system for identifying future errors. Whether or not data bears this out is another question, and there’s no clear answer given that most experiments aren’t very clear about exactly what teachers aim to do with corrections. At the very least, however, I can accept that some forms of correction might not be useful for learners in general, and it’s worthwhile trying to figure out which forms of correction, if any, are helpful.

4 Likes
207

The methodology of those statistics is less concerning to me than the actual exam itself since it’s a competency exam labeled a ‘proficiency’ exam. What I got most out of that self-eval notice is that JLPT freely admits that their exam isn’t a measure of proficiency :rofl:

1 Like
208

I’m having a hard time understanding what you’re getting at here. The definition of proficiency is literally “a high degree of competence or skill; expertise.”

Certification exams rarely guarantee competency / proficiency because the testing process would have to guarantee against cheating / brain dumping and other flaws present in virtually any testing methodology.

Think about it: college degrees make zero “guarantees”, even after years of education and expenses. Degrees and certification exams are just the best thing we have in lieu of direct observation of someone performing the desired task. You get from them whatever you put into them.

209

Yeah, anecdotally at least, I would love grammar corrections, as I need some way of realizing I’m doing something incorrectly.

Like, somethings will obviously be figured out over time (ie: major conjugation mistakes). But subtle things (ie: whether そうだ or らしい is more appropriate) are things I won’t have ways of figuring out other than happening to notice when reading a grammar book. Even natives have to learn it in junior high!

3 Likes
210

This study (I have access through my university) has this very interesting part:

Only when the grammatical errors were corrected directly did pupils show evidence of a long‐term learning advantage. Based on these findings, we conclude that both grammatical and nongrammatical errors are amenable to CF but that they benefit from different types of corrections: Direct correction is better suited for grammatical errors and indirect correction is better suited for nongrammatical errors. This leads us to suggest an alternative explanation for the apparent lack of a learning effect of comprehensive CF in Truscott and Hsu’s (2008) study

About the generalizability, they say themselves that
a) This study was conducted on students at an early age (learning Dutch at L2) => very specific context
b) The instructions weren’t provided by the teacher but the researcher
c) Feedback was provided only once
d) They compared two texts that students wrote and their error rate => might not be the best measure
…

In the end:

Hence, the present study does not support Truscott’s (1996, 1999, 2004, 2007) claim that written error correction is ineffective and can be even harmful. We conclude that comprehensive CF is a useful educational tool that L2 teachers can use to help learners improve their accuracy in writing.

I think it is very hard to conduct empirical studies on L2 acquisition as there are many contributing factors that are outside the researchers influence, especially when you want to measure long term effects. As said, it is good to know that this study (and the others) exist but we can’t jump to generalizable conclusions whether corrections are helpful or harmful or not.
Maybe this gets better researched in the future and the direction is clearer, but right now, a ultimate statement isn’t possible.
——————————
From my experience, corrections DO help, especially when they came directly after I said something. Like, I used から and ので interchangeably because I heard both for "because“ and I was never aware of the correct context usage until my Japanese tutor (L3) pointed it out. Also for my L2, English, there are a few words which I got corrected on by friends years ago, e.g. I used to say “fishes” for the plural of fish. Now, when I talk about several fish, I always have this voice of my friend in my mind, “not fishes, fish” :sweat_smile:
I never make this mistake again. Thus, I learned that FOR ME, they work and that’s why I ask my tutors to correct me. Idk, without feedback I would have used also my L1 (German) wrong, like in school, when I got corrected in one of my essays where I didn’t capitalize a verb though it was used as a noun in this context.
If one finds out, for themselves, that they don’t help you, well, then it’s also fine. People are different and one needs to find out what helps and what doesn’t.

I mean, I get that Refold wants that one is able to recognize these mistakes themselves and thus have some self-correction but it is a long process and even then, you won’t be able to spot them all and it’s easy to think you are highly proficient when no one points out that you actually aren’t. And it’s easy to get into that trap, especially when you are more advanced as your mistakes won’t necessarily kill the conversation, but you still sound unnatural (e.g. if you mess up articles in some languages, like “der, die das” in German or “le, la” in French).

3 Likes
211

That’s interesting! Just one thing: there’s actually an exception when saying ‘fishes’ is more common – if you’re talking about different kinds/species of fish. Example from Oxford Dictionary – ‘freshwater fishes of the British Isles’. You use it when you want to emphasise that there are different types.

Thanks for summarising the study, by the way. I find that corrections help me too, but I guess ultimately, the fact is just that everyone is different. There are probably some things that all human beings have in common when they learn, but there will always be specifics that make us unique, like our cultural background or life experience, and those things will change the details of what works best for each of us. :slight_smile:

1 Like
212

There are a number of mistakes in my native language that I used to regularly make; My mother was correcting me all the time. Those persisted for years until I consciously remembered her correcting me and stopped myself mid-sentence to correct it. As I corrected myself more and more, I’ve stopped making those mistakes.

So at least using myself as primary research subject, I think the whole study lacks a bunch of factors to consider.

At the very baseline there’s an inevitable truth: you’ll never fix your mistakes if you don’t know you’re making them. If nobody ever corrects anybody else’s mistakes, “correct” wouldn’t even exist in a definite sense.

But it seems that simply correcting people is not enough to trigger learning. It seems like there needs to be a conscious effort done towards fixing the problem by the subject themselves.

I’d like to add some more observations from the time I was learning to play piano. Stick with me for this analogy.

One of the things about practicing piano is that if you make a mistake and play a part wrong, you immediately notice it, in a hard “bang”/“board-scratching” way. But if your hands get used to playing it wrong a few times, it becomes quite challenging to change it again - you’d subconsciously play it wrong five, ten times - possibly even starting to annoy and frustrate yourself. It takes a directed mental effort to force your hands to play it differently, maybe still not correctly, but differently.

Later on, when I showed my performance to my mother, who also used to play piano, she pointed out that one of the parts that I played was incorrect. In this case, my “wrong” playing was playing a nice tune, just not the right one. I probably never would’ve noticed if not for her, but I checked with the sheets and she was indeed right, and I re-practiced that part since.

Now what I’m trying to get at with this whole side story is that it seems that for many skills, practice must involve both realizing a mistake and making an effort to fix it, and “correction” seems to be just one part of the formula, “mental effort” being the other. The latter is quite hard to quantify, however, so the lack of research into it is unsurprising.

4 Likes
213

If you want to break down the semantics then ‘proficiency’ alone is too broad. Look up ‘language proficiency’, first Google hit is ‘Language proficiency is the ability of an individual to use language with a level of accuracy that transfers meaning in production and comprehension’

Well, it’s got half of it…however since JLPT has zero measure of productive output, it’s a failed platform of proficiency measure. Besides, you shouldn’t even need a self-assessment survey on perceived proficiency, the test should be able to answer this more accurately. If I look at sources like TOEFL or CEFR, writing and speaking is always an integrated measure.

I know what your saying and I agree, there are limitations to what a certification can guarantee. But if you can pass N1 without basic communication skills (which is possible), there is a significant gap that is not addressed. But then again, if you just want to measure input skills then JLPT is fair to say an accurate measure of comprehension.

2 Likes
214

Thank you for your thoughtful reply. I agree that the general use of “proficiency” is quite broad, and people will often take different meanings from it (like what happened here). I think the JLPT site would be well served by simply stating something like “We test an individual’s Reading, Listening, and Grammar skills in the context of the following topics:” and then listing domains of understanding for each certification level.

Unfortunately, for now it’s the best we have. I appreciate that there are scalability and cost concerns with having someone administer and grade an oral and written evaluation, so I’m not sure if it’s possible that this will change much.

Somewhat on topic, if reading and listening comprehension is primarily what you need to develop to become fluent, the JLPT’s testing methodologies are quite comprehensive, are they not? (Tongue firmly in cheek, in case it’s not obvious) :rofl:

1 Like
215

Funny you say that, for me it was the exact opposite. All those explanations didn’t help at all, but through immersion it’s quite natural now what the nuances are (not that they even can be fully explained in English).

This is kind of like why I didn’t worry about it too much until I could read and listen about it in Japanese :slight_smile:.

Well put. I think the fundamental part again comes to “learning to notice”. Someone correcting you is not sufficient by itself; and a lot of the time you will start noticing your own mistakes from copying natives speech.

I just had this weird though, that maybe it’s actually on purpose. After all, it’s quite hard to develop advanced communication skills outside of Japan. For working purposes, having input certification is sufficient proof for companies, that these people are capable of developing their communicative skills quite rapidly. Then N1 true purpose would be basically just to advance the purposes of companies and immigration (because as a language test it fails as a whole).

Fun fact, a teacher I know listened to N1 listening comprehension, and said that it sounded super fast. But I guess it’s a well known fact tests are artificially made more difficult with weird tricks, like confusing answer choices.

3 Likes
216

While I intuitively feel that the “ability to notice” is beneficial too, big names like Stephen Krashen greatly emphasize that “deliberate notice” absolutely doesn’t matter (much less getting corrections). What matters is enjoying what you’re reading/listening so much that the language goes to the background. He even managed to convince Steve Kaufmann of this, or rather, Steve came to the same conclusion not long ago (here), after a very long time of being one of the preachers that “noticing” is a pillar of language acquisition.

My mind still tells me “noticing” is important, and part of me wants it to be true because that’s what I’m best at… but… These 2 names are the ones I respect the most when it comes to language learning, so I don’t know.

2 Likes
217

I can agree with you there. Guess you can leave the noticing for your brain to do subconsciously ^^. More and more I think language learning just happens as a subconscious process. At least it makes my learning a lot more stress free and fun.

Thanks, I also follow Kaufmann, but hadn’t seen that video!

2 Likes
218

It’s an interesting point. Far as I know though, there is no evidence that I’m aware of that communication skills develop rapidly upon employment for N1 passers, I suspect it’s very mixed. In fact JLPT may actually be holding back immigration efforts and is more of hindrance for lack of filtering out qualified applicants. So one has to wonder what it is actually accomplishing other than being the oldest testing standard and as a default, the most accepted.

Ministry of Justice Immigration Services Agency of Japan recently created the JPT test to satisfy the labor shortages and because JLPT is not designed at all to assess the language skills needed by workers and in their words mostly an ‘academic test’. This is a fairly new test only avail in East Asia but sort of the Ministry’s way of saying that the JLPT doesn’t serve the purpose they need.

So it’s still a reading/listening test but seems they refined it to be real world applicable. How this compares to BJT, I have no idea but at least BJT has some measure comparing JLPT which would assume to be more ‘real world’ business applicable given the nature of the test. As you can see the wide spectrum of N1 passer in relation to BJT scores is all over the map.

image
image846×497 47.4 KB

https://www.kanken.or.jp/bjt/english/about/feature.html

All three of these test are still input comprehension measurements; just a different focus and BJT appears to be at least 50/50 focus on listening/reading. For JLPT you really have a distinct advantage if you already know Chinese characters extremely well. 2/3 of the exam is reading and of that reading section, a minuscule portion actually tests correct kanji reading accuracy; the rest is mostly meaning comprehension. Of course you can’t fake grammar and listening but it can’t be ignored there is a lopsided nature to this test which relies heavily on the speed of one’s reading comprehension.

2 Likes
219

Interesting!

Looking at the chart of the nationalities at Japanese-Language Proficiency Test - Wikipedia, I think it’s safe to say that over 1/3 of the test takers are Chinese, who can use their kanji knowledge to rapidly extract information. In JLPT I feel that “skimming comprehension” is a lot more important. I wonder if a lot of the N1 passers are Chinese natives, who lack a in the listening/talking/comprehension department. Anecdotally I’ve met several Chinese N1 passers, whose communicative skills were absolutely ZERO. I guess they also lacked confidence, but still.

3 Likes
220

Statistically most of the JLPT takers (and thus pasers) come from east Asian countries. IIRC they make up 90% plus. Although that is spread out along East Asia.

221

I’m not saying this is wrong, but it’s with noting that Krashen does not equal the entire LA research community. I would be curious if that appears to be a widely held position.