Testing GPT-4.0 accuracy rate on Japanese language

It mostly stitched together sentences from the original article no? There is not a lot of room to get wrong.

If you want some example in which GPT4-0 goes off the road, here are two of them. It’s pretty representative of what happen in more difficult sentences IMHO, ChatGPT will say many right things, many wrong things, all with an authoritative tone.

From the senryuu thread:

Translate the following senryuu, break down the grammar and explain the humor: お噂は聞いていますと吹き出され


This Japanese senryū translates to: “I’ve heard the rumors,” you said, making me burst into laughter.

Now let’s break down the grammar:

  1. お噂 (o uwasa): This translates to “rumor”. In Japanese, “o” is an honorific prefix that makes the following word more formal or polite.
  2. は (wa): This is a particle used after the topic of the sentence, in this case, “rumor”.
  3. 聞いています (kiiteimasu): This is the present continuous form of the verb “to hear” or “to listen” (聞く - kiku). This implies that the action (listening/hearing) is currently ongoing.
  4. と (to): This is a quoting particle that is often used before verbs like “say”. In this context, it’s used to indicate what the person said.
  5. 吹き出され (fukidasare): This is the passive form of the verb “to burst into laughter” or “to spurt out” (吹き出す - fukidasu). The use of passive voice here emphasizes the action’s impact on the speaker.

Now, regarding the humor: The humor in this senryū relies on the implicit contradiction or irony. The act of someone mentioning they’ve been hearing rumors isn’t usually something to make one laugh out loud. However, the laughter could suggest the speaker finds the situation absurd, maybe because the rumors are wildly inaccurate, or because they find it amusing that the other person believes them, or perhaps even because the rumors are actually true but ridiculous. It’s up to the reader’s interpretation.

It’s rare, but even the English translation seems a bit weird to me. "I've heard the rumors”, you said, making me burst into laughter. Who is laughing here?

It’s even more confusing with the explanation about the passive: The use of passive voice here emphasizes the action's impact on the speaker. Who is “the speaker” ? The person telling the senryuu ? The person saying “お噂は聞いています” ? Are they the same? (In reality it’s a suffering passive, someone said “I’ve heard a lot about you” to the senryuu author while doing a 吹き出す)

This implies that the action (listening/hearing) is currently ongoing. The use of “currently ongoing” is a bit unclear no? It make it sounds like it’s a ている about what is going on right now, like how 食べている can mean “I’m currently eating”.

The entire explanations about the humor is completely wrong and again it’s very unclear who ChtGPT think is doing the 吹き出す.

Please translate the following sentence and break down the grammar: そりゃ話に乗ってくると取っていいんだな?


The Japanese sentence “そりゃ話に乗ってくると取っていいんだな?” can be translated into English as “If they go along with the story, it’s okay to take it, right?”.

Let’s break down the grammar:

  • そりゃ: This is a colloquial contraction of “それは”, which means “that is”. However, in this context, it’s being used as a filler word, similar to how “well” or “you see” might be used in English.
  • 話に乗ってくる: The term “話” means “story” or “talk”, “乗る” means “to ride” or “to board”, but in this context, it is used metaphorically to mean “go along with” or “agree to”. The “に” particle indicates the direction or target of the action, and “くる” is a helper verb that implies the action is coming towards the speaker. So “話に乗ってくる” means “to go along with the story”.
  • と: This is a conditional particle that is similar to “if” or “when” in English.
  • 取って: This is the te-form of the verb “取る”, which means “to take”. The te-form of a verb is often used to connect two clauses or to express a conditional relationship.
  • いい: This adjective means “good” or “okay”. When combined with the te-form of a verb, it’s used to express permission or acceptance, similar to the English phrase “it’s okay to”.
  • んだ: This is a contraction of “のだ”, which is a sentence-ending phrase used to provide explanation or emphasis. Here, it can also imply that the speaker is seeking confirmation or agreement from the listener.
  • な?: This is a sentence-ending particle used to seek confirmation or agreement from the listener, similar to “right?” or “isn’t it?” in English.

So, putting it all together, “そりゃ話に乗ってくると取っていいんだな?” translates to “If they go along with the story, it’s okay to take it, right?”.

Not completely wrong, it correctly say that 話に乗る is an idiom, but the sentence is supposed to mean something like "So can I take that to mean you agree / you are in ? (about something they talked earlier, probably)

と: This is a conditional particle that is similar to "if" or "when" in English. Doubtful, seems more like the quotation と to me.