Evaluate your skills in the language

Japanese Language
1

Hi everyone,
I wanted to bring up to attention an article that was posted recently in the MIA website (the new AJATT like community). Specifically this time regarding a proposed method to evaluate skills in language.

If your are not aware of AJATT maybe this will help

A little of context maybe. To those who don’t know. AJATT (All Japanese all the time) was a website that proposed a methodology in which you immerse as much as possible from day 0 in the language (japanese in that case). quickly going through the book Remembering the Kanji initially (hopefully in some 3-4 months period) and quickly reviewing grammar as well (a month or so)… then immersion and sentence minning (picking and reviewing sentences that contain no more than 1 unknow concept) been the keystone for gradually, but quickly, aquiring proficiency in japanese ( the author of the website claimed in his case that was 18 months).

Anyway, the author of MattvsJapan channel on Youtube, who has done a lot of videos regarding AJATT one the past, has been doing some posts recently on his website and Patreon account on how to put order into this process and explain the individual parts of it; so it actually has change the name of AJATT to MIA, once the alterations were enough to justify a new name (and also to avoid direct vinculation with whatever negative comment regarding AJATT and the developer of the website came to have at one point in the japanese language community)

Yesterday the author of the MIA website (Matt) posted this model to evaluate your competences in the language, alonside with some example of Youtube videos, which were evaluated using the proposed scale.

I’m really interested in this, as I’ve avoided the JLPT way as a method to test myself (it has no value personally in my projects with the language; and the “I passed N1 but can’t function in the language” stories serve as a warning that the test it’s measuring something… just not what many may anticipate).

I will be expecting the next posts regarding this subject and an actual way to make such evaluations in my case, as once you go out of the more academic way to learn japanese, including the material you use, it’s so easy to feel loss in terms of actual progress. So this might shed some light in that regard.
Whatever I’m doing to learn japanese this days it’s much more in sync with AJATT than anything else actually; but still I know there’s people here with lots of experience regarding learning japanese, so I’m always wanting to hear any caveats you might pick :+1:

If you aren’t aiming for JLPT, do you have a way to actually test your skills?

As a reference… I think my english speaking skills are somewhat in between the 4th and 5th example. … In japanese I don’t think I’m much better than the first example provided :man_shrugging:

4 Likes
2

As long as it doesn’t turn in to “I’m totally self-assessed fluent” but can’t pass N3 :wink:

If you want to do something serious with the language (like finding a job in Japan) there is probably no way around a standard test, even if it makes little sense.

8 Likes
3

I’m not done reading it, but one thing that sticks out, as far as I am concerned, is that there’s no reference to actual research on second language acquisition. :thinking:
As the author said, a finer-grained description of language capabilities is important in that context, so I assume that people gave thoughts to that before. Regardless of the quality of his model, it would be nice to have a comparison with the actual state of the art. (I don’t know, maybe stuff like the CERF is the state of the art, but I doubt it)

7 Likes
4

The text is quite long, does it say how you actually find if someone is using 5 points in grammar or 6? How do you objectively do that?

1 Like
5

Who is supposed to be doing the evaluations? If people are supposed to evaluate themselves, won’t their lack of general language ability limit their ability to accurately assess their own level? The Dunning-Kruger effect.

Until the other day I thought I had the correct pitch accent for 授業, a word I say about 10 times a day, since I’m a teacher, and it turned out I was wrong.

13 Likes
6

It’s interesting that this author decided to come up with their own model for assessing language proficiency / fluency when there already exists a number of proficiency-based scales through national and international organizations, like the American Council on the Teaching of Foreign Languages (ACTFL) and Common European Framework of Reference for Languages (CEFR).

Below you’ll find links to these documents / websites, explaining the varying scales:

I think the above should act as a way of self-assessing language proficiency as it’s backed by professional organizations and research. And if you’re looking for a way to test your proficiency, you could try the Stamp Test, which aligns with the ACTFL Proficiency Scale.

Hope this helps!

4 Likes
7

I quickly skimmed through but my thoughts on this are like acm said. Self evaluation is nice and all, but take it with a grain of salt. A lot of grains of salt.

I also have no interest in the JLPT for assessment of my skill, so I’m in the same boat as you, but remember that self assessment shouldnt be your go to on its own. Having actual japanese people correct your speaking and give you feedback seems far more valuable than assessing yourself with a rubric that this guy named Matt made. Its not a bad idea to maybe get a general idea, but I don’t think this is an improvement on anything personally except for maybe in the convenience aspect.

yeah, right here 6-Point Rubric - Google Sheets

4 Likes
8

It’s just some wish list judged by the language god :wink:

9

Changing now would be admitting defeat! You must pretend it was a conscious choice and continue into perpetuity.

6 Likes
10

I don’t have any insight to the method posted by the OP, but just wanted to point out that the CEFR is recognised as not being particularly translatable across all languages; it is the European Framework of Reference, after all. It doesn’t really take into account things like the acquisition of different alphabets, for example.

The language school I attend uses the CEFR to rate the levels of many of its languages, but deliberately does not use it for e.g. its Japanese, Chinese, Arabic courses (or at least pairs it with something else).

I do agree that a framework backed by research would be something you could use with a little more confidence.

6 Likes
11

@nbeck0212
Thanks I’ll look into some of those… I was aware of the CEFR, will look into the other ones. I’ll deffinitly look into the Stamp Test. Thanks :+1:

Yeah.I know the whole model it’s an opinion at best. I think it’s in sync with my own opinions, but I’m clear I’m not seeing language learning rewritten before my own eyes also :blush: …

Yeah, I think there’re nuances that aren’t taken into consideration in the CEFR when used as a measurement tool for japanese.

I’m thinking in the end to use something among these lines. Having an iTalki japanese teacher and send him the scales alongside with some videos of people talking at different levels (maybe the same ones). And then over the year send the teacher my own videos with a 3-6 month span in between. I most likely will trust in a japanese teacher and then see how much it correlates with my own perception using the same tool.

I expect to interact with people there, learn more about japan and some particular hobbies I’m currently interested soon enough. In the end I think professionaly I would like to have some connection with japanese people, more as a colaboration than anything else. So, yeah while I expect to do serious things with the language… still I expect the test to go unrequired.

1 Like
12

I’m trying to check my comprehension level with that spreadsheet, if “moderate” is already “college lectures, the news, fast comedy, old movies” and “difficult” is “high-level lecture by a particularly verbose college professor, a four-decade old historical period drama with archaic language and poor audio quality, or a muffled conversation recorded on a damaged cassette tape” I doubt most natives exceed 6 points in comprehension :slight_smile:

4 Likes
13

The main use I see of these kinds of frameworks is by teachers and assessors, so that sounds like quite a sensible approach (although they might not have the patience to sit through videos, or learn a new framework if they’re already familiar with another). That said, our teacher occasionally asks us to rate ourselves against a framework because she finds it helpful to understand how we perceive our own language ability.

I do think self-evaluation can be a helpful exercise, assuming you approach it in the knowledge that you’re not the best judge. Even just looking at a framework and understanding what speakers at different levels would be expected to manage can be interesting.

But yes, I’d be interested to know if there’s a more rigorously-produced framework out there for Japanese or perhaps a broader group of languages.

4 Likes
14

the problem is that everyone’s an expert, even if they have no expertise. so many people “know” so much about the language, without knowing the language, so i tend to just ignore such stuff.

there’s good stuff to be read and learned from, from renowned real experts like Krashen, i really don’t need the “2c” from any random hobo on the net.

4 Likes
15

You don’t need N2 to get a job in Japan.

16

You don’t even need Japanese to get a job in Japan :wink:

3 Likes
17

I know what you mean. But I think what use you can make out of material that some “random hobo” produces it depends only on how do you take it into account.

I’m conscious that there’re people doing proper research on the matter, and surelly that body of work should account for more weight into the decisions I make to learn about that field. But then, knowing enough about the “random hobo” you can take his opinion into account as well (sort of like the level of evidence pyramid that it’s used for scientific research).

I totally expect people reading this will question what is presented and after filtering the rubbish, something still can come through :wink:

1 Like
18

Actually the graded examples were probably what I liked the most about the article.
I would pretty much like this kind of example would be presented alonside with different scales. Would be such an encouraging addition to know what to aim for.

1 Like
19

If you are doing the evaluation just for yourself, what is your goal? For a test you need someone with a perfect skill to judge the correctness, which you probably don’t have at hand.

Wouldn’t it be enough for example to take a text for reading comprehension and see what you can make of it, or talk to someone and measure the degree of confusion on the face? These methods want to reach a level of objectivity that seems very hard to achieve, and is not really necessary for your purpose anyway.

If you can read what you want and say what you want you would already be in a great position.

How do you demonstrate 10 points in content (Has ideas or messages powerful enough to majorly impact the lives of many people, or even bring significant societal change.)? :slight_smile:

2 Likes
20

Just adding to the discussion:

In case anyone is wondering if there is a nice framework for Japanese based on actual research, there is a real study group for applying OPI to Japanese who has been active for quite a while. Their website is totally Web 1.0 (but that is real Japan, right?), but it does have a lot interesting material detailing the framework and how tests should be applied (since it is oriented to teachers and researchers).

The framework is just for oral proficiency (duh), but it is divided in 10 levels with an extra one on top of everything.

They also link to this really nice page from ACTFL that has (short) audio examples for each level with the Rationale for Rating for each one. Although everything is in Japanese, for some reason the Rationales are in English, go wonder.

However, they are not really promoting regular exams in Japan nor anything, so it’s not like you can easily take (although not impossible) an OPI exam even if you want. But it is nice to see a framework based on real research and all.

4 Likes