Getting a list of all wanikani kanji - known and unknown

Hello, I’m going to try my hand at making a few API calls with wanikani so I can make some data visualizations around my learning. I have only done API calls a few times, so I will probably have questions about that when the time comes. For now, is it possible to get a list of all the kanji you’ve burned, are learning, or have yet to learn and have them be organized as such? Sorry if it’s such a basic question and thank you!

It’s been a while, so I’m a bit rusty, but it might be worth to check @seanblue’s SRS and leech breakdown script. In there, he has some code that counts all the kanji that are in a certain SRS category, and I’m pretty sure it’s possible to do this for the Lesson and Unlisted (or something) categories too.

I’m not clear whether your goal is to learn how to use the API, or whether you want to look at your kanji?

If the latter, it’s always worthwhile to check out www.wkstats.com which shows you all the gory details :wink:

3 Likes

I think you should skim the WK API reference:

https://docs.api.wanikani.com/

Even without using it, you get a good idea of what data is exposed and how it’s accessible.

2 Likes

For this kind of things you can use the export feature of Item Inspector.

Yes, you need to get the assignments data structure:
https://docs.api.wanikani.com/20170710/#get-all-assignments
then you can use its fields to split to split into the categories you are asking for:

  • use subject_type attribute to filter by items type (kanji, radical, vocab)
  • burned items will have burned attribute set to a burn date (not yet burned have it set to null)
  • started_at will be null if you haven’t completed a lesson for this item, other wise it will have a timestamp of when you completed the lesson

You can query all assignments and categorize them locally or you can query only assignments you want by using relevant query params.
E.g. get only burned kanji (I think it should work but I haven’t tested it):

GET https://api.wanikani.com/v2/assignments?subject_types=kanji&burned=true

Don’t forget to handle pagination. WK API returns up to 500 items per page. Your code will need to automatically fetch the next page of results until next_url is null.
https://docs.api.wanikani.com/20170710/#pagination

To see the actual item you’ll need to join the assignments with subjects using subject_id.
https://docs.api.wanikani.com/20170710/#subjects

Send a few requests, check out the data and you’ll figure it out.

If you have more specific questions about querying the API later on it’d help if you share your code snippets.

1 Like

Thanks this was great! I can get the assignments, but what I actually want is a list of the specific kanji. At a high level, I want to know this:
道:burned
寿:not learned

Where I get the specific kanji and my progress in it. Is this possible?

You could use the export feature of my script Item Inspector to get exactly this.

If you export the SRS stage you will know whether the item is burned or not learned (SRS is Locked or Initiate).

2 Likes

As I wrote above you can get all that information by fetching the assignments and subjects data. From the assignments data you can extract the state of the item (i.e. learned, burned etc). But to understand which kanji it is you need to join it with the subject data.

For example, here’s an assignment object:

{
  "id": 80463006,
  "object": "assignment",
  "url": "https://api.wanikani.com/v2/assignments/80463006",
  "data_updated_at": "2017-11-29T19:37:03.571377Z",
  "data": {
    "created_at": "2017-09-05T23:38:10.695133Z",
    "subject_id": 440,
    "subject_type": "kanji",
    "level": 1,
    "srs_stage": 8,
    "unlocked_at": "2017-09-05T23:38:10.695133Z",
    "started_at": "2017-09-05T23:41:28.980679Z",
    "passed_at": "2017-09-07T17:14:14.491889Z",
    "burned_at": null,
    "available_at": "2018-02-27T00:00:00.000000Z",
    "resurrected_at": null
  }
}

You can see that this a kanji that you have studied (started_at is not null) and it is not burned (burned_at is null).

But you don’t know which kanji it is. So you need to get the subject data by subject_id.
Here it is:

{
  "id": 440,
  "object": "kanji",
  "url": "https://api.wanikani.com/v2/subjects/440",
  "data_updated_at": "2018-03-29T23:14:30.805034Z",
  "data": {
    "amalgamation_subject_ids": [
      56,
      88,
      91
    ],
    "auxiliary_meanings": [
      {
        "meaning": "one",
        "type": "blacklist"
      },
      {
        "meaning": "flat",
        "type": "whitelist"
      }
    ],
    "characters": "一",
    "component_subject_ids": [
      1
    ],
    "created_at": "2012-02-27T19:55:19.000000Z",
    "document_url": "https://www.wanikani.com/kanji/%E4%B8%80",
    "hidden_at": null,
    "lesson_position": 2
    "level": 1,
    "meanings": [
      {
        "meaning": "One",
        "primary": true,
        "accepted_answer": true
      }
    ],
    "meaning_hint": "To remember the meaning of <kanji>One</kanji>, imagine yourself there at the scene of the crime. You grab <kanji>One</kanji> in your arms, trying to prop it up, trying to hear its last words. Instead, it just splatters some blood on your face. \"Who did this to you?\" you ask. The number One points weakly, and you see number Two running off into an alleyway. He's always been jealous of number One and knows he can be number one now that he's taken the real number one out.",
    "meaning_mnemonic": "Lying on the <radical>ground</radical> is something that looks just like the ground, the number <kanji>One</kanji>. Why is this One lying down? It's been shot by the number two. It's lying there, bleeding out and dying. The number One doesn't have long to live.",
    "readings": [
      {
        "type": "onyomi",
        "primary": true,
        "accepted_answer": true,
        "reading": "いち"
      },
      {
        "type": "kunyomi",
        "primary": false,
        "accepted_answer": false,
        "reading": "ひと"
      },
      {
        "type": "nanori",
        "primary": false,
        "accepted_answer": false,
        "reading": "かず"
      }
    ],
    "reading_mnemonic": "As you're sitting there next to <kanji>One</kanji>, holding him up, you start feeling a weird sensation all over your skin. From the wound comes a fine powder (obviously coming from the special bullet used to kill One) that causes the person it touches to get extremely <reading>itchy</reading> (いち)",
    "reading_hint": "Make sure you feel the ridiculously <reading>itchy</reading> sensation covering your body. It climbs from your hands, where you're holding the number <kanji>One</kanji> up, and then goes through your arms, crawls up your neck, goes down your body, and then covers everything. It becomes uncontrollable, and you're scratching everywhere, writhing on the ground. It's so itchy that it's the most painful thing you've ever experienced (you should imagine this vividly, so you remember the reading of this kanji).",
    "slug": "一",
    "visually_similar_subject_ids": [],
    "spaced_repetition_system_id": 1
  }
}

The kanji is available at the attribute characters. The meaning is an element in the meanings array where primary is true. Same with reading but use the readings array.

This is awesome! How did you go about “joining” it? Not sure if you have any script that I can look at for the API call. Thanks so much for your help and explanations.

The specific method of “joining” the data is up to you really. The API doesn’t provide this, so you need to request the assignment and subject data separately and then connect the data points in your code. How to do it will vary greatly depending on your lanaguage and library of choice.

For example, if you used Python library pandas there’s literally a method called join(). pandas.DataFrame.join — pandas 1.5.0 documentation However, if you have no experience with pandas it’s not very intuitive.

Which programming language are you using?

I’m using javascript - just in VS code

It’s been a while since I used JS but I found some of my old code querying WK review data. Hopefully it helps, but I’m not sure it will work if you try running it as is. I wrote this code using ObservableHQ notebook and it has its peculiarities:

Define helper functions

getWkItems = async function fetchAllReviewStats(
  endpoint = "review_statistics"
) {
  let reqOpts = {
    method: 'GET',
    headers: { Authorization: `Bearer ${Secret("WK")}` }
  };
  let resuts = new Array();

  let nextUrl = `https://api.wanikani.com/v2/${endpoint}`;
  while (nextUrl) {
    let resp = await fetch(nextUrl, reqOpts);
    if (resp.ok) {
      let body = await resp.json();
      nextUrl = body.pages.next_url;
      resuts.push(...body.data);
    } else {
      throw Error(`${resp.status}: ${resp.statusText}`);
    }
  }
  return resuts;
}

Get review data:

  const reviewStats = new Map();
  const data = await getWkItems("review_statistics");
  data.forEach(d => {
    reviewStats.set(d.data.subject_id, {
      updated_at: d.data_updated_at,
      ...d.data
    });
  });

Get a map of Subjects:

  const subjects = new Map();
  const data_results = await getWkItems("subjects");
  data_results.forEach(d => {
    subjects.set(d.id, { updated_at: d.data_updated_at, ...d.data });
  });

Sort data:

  let sortedReviewStats = new Map(
    Array.from(reviewStats.entries()).sort(
      (e1, e2) => Date.parse(e2[1].updated_at) - Date.parse(e1[1].updated_at)
    )
  );

Connect Subject data to sorted Review data:

 const joinedData = new Map();
  sortedReviewStats.forEach((d, id) => {
    joinedDarta.set(id, { ...d, subject: subjects.get(id) });
  });

this is the statement that looks up the subject by its id
subject: subjects.get(id)

Upd: updated some variable names for clarity

You can see that the most common method I’m using is forEach() which accepts an arrow function that will run on every element of your data.

1 Like

Thank you so much I’ll test this out! Do you do data viz too? I use ObservableHQ almost daily :grin: The reason why I want this data is so I can visualize it in my own way.

Oh wow, didn’t expect you used Observable too :smiley: It’s such a niche product!

I’m a data engineer by profession, so I mainly work with data collection and processing, as well as various backend development and cloud infra. But I do data viz occasionally. Less and less now though.

I used Observable to learn D3.js about a year ago. Played around with it and moved on :slight_smile:

So cool. I work with data too, but more on the design/storytelling side. I’m trying to learn more data science and analytics, though.

I edited the code to get it to work and combined the two calls. I thought I would post it here incase anyone else is curious. The next thing I’ll do is take out all of the keys/values that I won’t be using.

const apiToken = "<api_token_here>";
//define helper functions
const getWkItems = async function fetchAllReviewStats(
	endpoint = "review_statistics"
) {
	let requests = {
		method: "GET",
		headers: { Authorization: "Bearer " + apiToken },
	};
	let resuts = new Array();

	let nextUrl = `https://api.wanikani.com/v2/${endpoint}`;
	while (nextUrl) {
		let resp = await fetch(nextUrl, requests);
		if (resp.ok) {
			let body = await resp.json();
			nextUrl = body.pages.next_url;
			resuts.push(...body.data);
		} else {
			throw Error(`${resp.status}: ${resp.statusText}`);
		}
	}
	return resuts;
};

//get review data

const joinData = async () => {
	const reviewStats = new Map();
	const getReviewStats = await getWkItems("review_statistics").then((data) => {
		data.forEach((d) => {
			reviewStats.set(d.data.subject_id, {
				updated_at: d.data_updated_at,
				...d.data,
			});
		});
	});

	//get map of subjects
	const subjects = new Map();
	const getSubjects = await getWkItems("subjects").then((data) => {
		data.forEach((d) => {
			subjects.set(d.id, { updated_at: d.data_updated_at, ...d.data });
		});
	});
	//sort data
	let sortedReviewStats = new Map(
		Array.from(reviewStats.entries()).sort(
			(e1, e2) => Date.parse(e2[1].updated_at) - Date.parse(e1[1].updated_at)
		)
	);

	//connect subject to sorted data view
	const joinedData = new Map();
	sortedReviewStats.forEach((d, id) => {
		joinedData.set(id, { ...d, subject: subjects.get(id) }); //this looks up subject by its id
	});

};

joinData();
2 Likes

This is pretty embarrassing, but I was hoping you’d be able to help. So I was able to make my viz in observable, but I’m trying to bring it into vscode, and I haven’t worked in vanilla js & node in so long, and I haven’t been able to get the object in a variable - only when I console.log. Each try as resulted in a “pending Promise.” Do you have an idea how the resolve this? I looked all over stack overflow etc. and have not been able to find the solution.

Here’s the gist, which I believe is the same as what I posted above: Wanikani learning · GitHub
Sorry for all my n00b issues. This is really killing me.

Sure, I’ll see if I can help but probably tomorrow.

Just some thoughts in the meantime:
As far as I remember the fetch api may be causing an issue because Observable is running it sequentially but it’s an async function normally.

I think thare are other libraries that you can use in node.js for sending requests synchronously.

1 Like

Thanks so much I appreciate it! Yes I know how to do these things in Observable and other frameworks but not in vanilla js…and for whatever reason I just cannot figure it out (TBH two of my other dev friends were confused too so I’m not the only one who forgot!)

So the issue was just that I was calling it outside of an async function - I needed to save it and access it in an async function.

1 Like