Wanikani Open Framework [developer thread]

Oooh… A replacement for the existing one??? That would be cool.

(And if not a replacement, still probably cool :wink:)

3 Likes

Yep! It’s something that I have been tinkering on a bit here and there. I wanted a more advanced script that lets you define reordering patters so you don’t have to change anything during the review.

My review preference as an example:

  • I want to review in random order
  • But only 100 reviews per session
  • If there are more than 100 reviews available
    • I want to prioritize critical items (not that I have any anymore)
    • I want to prioritize lower level reviews
    • I want to prioritize radicals and kanji over vocab (within the last level)

That is possible with my script, by creating a sequence of sorts and filters which do those things (in reverse)

WIP interface

For quickly changing the active sequence

image

4 Likes

Working name is Omega Reorder, to up the ante from Reorder Ultimate 2

3 Likes

Sounds cool. I would definitely use it.

1 Like

Very nice! Almost exactly the sort of thing I had in mind: a list of prioritized rules as a preset.

2 Likes

2 Likes

What font is that?

3 Likes

oh gosh, I’ll see if I can find it

Edit: Found it!
image

3 Likes

Thank you! I like it, it reminds me of ITC Kristen but less bubbly

3 Likes

Will this script also work for lessons? When I was using the reorder script during the faster levels, I would have preferred a rather complicated lesson ordering. Seeing your screenshot, I was wondering whether this order would be doable through your interface:

  1. All items with level ≤ current level - 2
  2. Radicals of the current level
  3. Kanji of the current level
  4. Radicals of the previous level
  5. Kanji of the previous level
  6. Vocab of the previous level
  7. Vocab of the current level

I understand if this is outside the script’s scope – or maybe it is already possible and I’m just not seeing it?

Some additional thoughts about this which are probably dumb and should be ignored

When I was pondering over this, I thought that it could work with actions of type “Filter” and “Fix + Append Others”. “Filter” is self-explanatory and it seems you already have it implemented. “Fix + Append Others” would fix the order in the current state so that these items are unaffected by subsequent actions, and then append all items that were removed by a previous “Filter”.

The order mentioned above could then be achieved by:

  • “Filter” Level ≤ current level - 2
  • “Fix + Append Others”
  • “Filter” Type = Vocab (Inverted, so actually Type != Vocab)
  • “Sort” Type
  • “Sort” Level (Inverted)
  • “Fix + Append Others”
  • “Sort” Level

Sorry for the unsolicited ideas.

1 Like

The plan is to do lessons eventually, but since I don’t have any lessons anymore it’s not a priority in the near future.

I don’t think all of that is possible with just one sequence in the current state of the script, but you could do it if you’re willing to change sequence once and let passed radicals and kanji of the current level be done between 6 and 7

You would have one sequence for #1, then the second sequence would be sorted by type, level, and critical

  1. Sort by type
  2. Sort by level (items are now ordered by type within levels)
  3. Sort by critical (brings not passed radicals and kanji of current level to front)

Interesting idea! That seems super useful. I’m too tired to figure out if it would work with the current architecture, but I think it’s definitely worth considering!

2 Likes

Oh, so that is the meaning of “critical items”! I thought you were referring to the (in my opinion totally useless) list of “critical condition items” on the vanilla WK dashboard.

Right, just changing between sequences is also a good solution.

I assume when starting a review session, initially no sequence is applied? So the workflow would be

  • Start review
  • Select sequence #1
  • Do all reviews
  • Get taken back to the summary page
  • Start review
  • Select sequence #2
  • Do all reviews

Another question: will you allow to sort the reviews by how much they are overdue? I think if one has a huge backlog, it might be useful to do the items that are less overdue first. With those you still have a chance to get them correct and continue with the SRS, while the others are likely already forgotten and have to be relearned anyway.

1 Like

The more I think about it the more I like this “Fix + Append” idea. It’s not as efficient but it’s a lot more versatile and makes the logic of creating sequences much easier.

I think typically a user will stick to one sequence, so the one last used is applied on load.

Yep, that is already implemented! You can either sort by overdue or filter with a percentage. I think I might also add a leech sort/filter

2 Likes

Great! :slight_smile:

The only problem I see with this is: What happens if you try to start your reviews, but the sequence does not contain any more items? Will you get kicked out of the review session immediately? How would I be able to switch from sequence #1 to sequence #2 in such a case? Is the sequence setting also available from outside the review screen? Or does the sequence filter reset if at the start of the session, no items are left when the sequence is applied?

Sorry that I’m so nosy – I just think this is an interesting script.

1 Like

I am aware of it being an issue (an honestly I have not tried to see what happens if you set current item, active queue, and review queue, to empty arrays), and have it on my todo to figure out. Maybe a popup asking if you wish to proceed without any sorting/filtering or be taken back is enough

Yep, that’s the plan! Although currently, it’s only running in reviews for convenience

No solution for this yet. I still have plenty to do before it’s ready

No problem! It’s great to have some more opinions

2 Likes

I’ve got a wild hair to write yet another user script, despite knowing full well that “maintenance is forever.”

I had figured out how to get most of what I was after, but I was struggling to obtain some information.

As usual, just writing down my questions clarified my thinking and helped me to provide my own answers!

I think I’ll post this anyway in case it helps anyone else. Please correct me if I’m mistaken about something or let me know if you have any further suggestions.

These were the four pieces of info that I was struggling to extract:

  1. The number of “items” (subjects) reviewed in the past <interval> (say, 72 hours). The same item may, of course, have been reviewed multiple times over that interval.

  2. The number of “sessions” involved in reviewing the items from 1. One session reviews multiple items.

  3. The total time spent reviewing the items from 1.

  4. The number of incorrectly answered items over that interval. (The number of items/subjects that didn’t get a correct response for both meaning/reading parts on the first try, not the count of incorrect answers.)

@Kumirei’s wonderful heatmap script (which uses review_cache) calculates and displays all of these over various intervals, but I was getting lost in the code trying to understand how they are calculated. In particular, I was struggling to understand how “sessions” and “reviewing time” were calculated.

If my newfound understanding is correct: Subjects are the unique r/k/v items available to study (always returned by any query to WKOF’s get_items()). Each subject is reviewed multiple times, of course. Each time an individual Subject is reviewed, a new Review item is created with the ID for that subject along with additional metadata (like the start time).

The API also provides access to a review_statistic object for each subject that has been reviewed at least once. If my understanding is correct, this object tracks the total correct/incorrect answers for that subject since starting with Wanikani, so isn’t useful for my purposes.

So I think I can get what I’m after as follows:

  1. @Kumirei 's get_reviews() function from Review Cache provides what I need.

  2. I see now that @Kumirei uses a heuristic to determine which reviews are in the same session. As long as a review starts within 10 minutes of a prior review, it is considered to be in the same session.

  3. I think, but I’m not totally sure, that “review times” are calculated similarly. The elapsed time from the start of the first review in a “session” to the start of the last review in the same session is considered “review time”.

  4. This can also be obtained from get_reviews().

Anyway, I think I’m now able to make progress. Apologies for the interruption!

2 Likes

Yep, all that you need is the reviews. WK does not store any session info, so you’ll just have to figure all of this out from the review data.

10 minutes is the default, but since this was a completely arbitrary choice on my end it is also a setting (“session time limit” in the “general” tab). If you change this setting you’ll see that the number of sessions and the total time changes

That is exactly how I do it

2 Likes

I’m trying a “show-stale-while-refresh” strategy in a script I’m writing that munges review data.

I’d like to have a continuously looping background process that periodically (say, once every 5 seconds or so) checks for new review data. Only when new data is retrieved is it processed and stored in a browser-side cache (triggering a reactive update to any visible elements on the page).

At first, I didn’t see the advantage of wkof.Apiv2.fetch_endpoint('reviews', options) over just calling the API directly with fetch(). After reviewing the code, I see that in addition to:

  • Adding the user’s API token to the request
  • Adding any desired filters to the request
  • Assembling paginated responses
  • Handling error responses from the server
  • Parsing the JSON response from the server
  • Rate limiting (if 429 from server, retry up to 40 times with gradually backed-off delays)

fetch_endpoint() also includes the If-Modified-Since header, meaning it supports:

  • Conditional requests (quick, lightweight responses from server if no new data).

(suggesting that I can safely use fetch_endpoint() in my continuous loop).

Since the If-Modified-Since header is present, I don’t think my plan will present too onerous a load for the server … at least in the happy path … if I code it properly.

I think “properly” means at least the following (but I’d like to hear from more experienced people here if I’m missing something):

  • Caching results in indexedDB (easy with file_cache)
  • Keeping track of the datestamp of the most recent review.
  • Only querying for data that is newer than the datestamp.
  • Ensuring each call to fetch_endpoint() resolves before looping (i.e. awaiting the response each time rather than looping with asynchronous requests). (To prevent queuing in the face of rate limiting.)

I think these last two might be unnecessary, but:

  • Ensuring the looping process is killed whenever the user navigates off the relevant pages (dashboard and review pages in this case). I think this is automatic but will verify.

  • Handling idle browsers. If the user leaves the browser on the dashboard page then goes away for a weekend without doing any reviews, there’s no sense in continuously pinging the server every five seconds for several days. The simplest solution is probably to implement a maximum number of loops (necessitating a page refresh after, say, 2 hours on the same page).

Thoughts? Anything else I’m missing?

Sounds like you’ve thought it through pretty well.

You are correct: All page activity, including open requests, will terminate automatically when the user leaves the page.

Another alternative for the “idle browser” is to:

  1. Check if the tab is active. I don’t know how it’s done, but I’ve seen sites do it.
  2. Look for mousemove events to reset a timeout timer.
2 Likes

Ooh. I like that one!