Turbo Events Library for Wanikani Open Framework

WaniKani API And Third-Party Apps
1

:warning: This is a third-party script/app and is not created by the WaniKani team. By using this, you understand that it can stop working at any time or be discontinued indefinitely.

What is this?

Wanikani Open Framework Turbo Events is a library for script developers to include in order to simplify their workflows when dealing with Turbo Events. If you are a bit out of the loop and unfamiliar with why this is now very important, see tofugu-scott’s post here and his announcement here for more background.

Essentially, pretty much any script that was not using a library that takes care of handling the turbo events (i.e., almost all of them) behaves unexpectedly after the update. Users are forced to perform a manual refresh of the page after they navigate anywhere in order to get basic script functionality, if they’re lucky.

And that’s where this library comes in. Developers, read on.

How does it help?

It abstracts away most of the guesswork and troubleshooting for dealing with the turbo events, while allowing sufficient flexibility for most, if not all, feasible usage. Adding a basic listener for turbo:load can be as simple as:

wkof.turbo.events.load.addListener(callbackFunction);

Or you could go a step further and easily make sure that the event happens on a certain url:

const reviewsRegex = /https:\/\/www\.wanikani\.com\/subjects\/review.*\/?$/;
wkof.turbo.events.before_render.addListener(callbackFunction, reviewsRegex);

Just be sure to include code to ensure that this is called after the Turbo Events library has been set up. This can be done conveniently with the wkof.ready(module_list) function or wait_state(state_var, value, callback, persistent) function. For example:

const setupTurboListener = () => wkof.turbo.on.common.dashboard(main);

wkof.ready('TurboEvents').then(setupTurboListener);
// or, this does the same as the line above
wkof.wait_state('wkof.TurboEvents', 'ready', setupTurboListener);

Or, if you’re already in an async function:

await wkof.ready('TurboEvents');
wkof.turbo.on.common.dashboard(main);

Take a look at one of the repositories below for more basic information, general notes, and examples.

Requirements

  1. WaniKani Open Framework
    • While it could be rewritten to not require the WaniKani Open Framework, my goal was to add something that could be used in a way that is convenient for development. Therefore, I decided to create it in a way that works as an addon to wkof. However, that is the extent of what is needed.
  2. Set the @match userscript directive to https://www.wanikani.com/* or equivalent.
    • Otherwise, the script may not end up running if the user refreshes the page somewhere unexpected.

Installation

As of writing, this is a library script; thus, there are limited ways of importing it.
Note that these example URLs might not always be the most up to date. Check the repositories section below for the current versions.

  • The preferred method would probably be something along the lines of the following:
const scriptUrl = 'https://update.greasyfork.org/scripts/501980/1426289/Wanikani%20Open%20Framework%20Turbo%20Events.user.js';
wkof.load_script(scriptUrl, /* use_cache */ true)
  • Alternatively, it can be @required in the script header like so:
// @require https://update.greasyfork.org/scripts/501980/1426289/Wanikani%20Open%20Framework%20Turbo%20Events.user.js

Repositories

For now, these are both manually kept in sync.

  • [GitHub]
    • The readme has code syntax highlighting here, which makes it much easier to read
  • [Greasy Fork]
    • You can get a version-locked link at the History link by clicking the desired version, copying the last part of the URL, and replacing the part after /scripts/501980/ in the update URL with the copied version number.

Credits

This was heavily inspired by the work of @rfindley and the discussions of @LupoMikti, @ctmf,
@Sinyaven, and others in the following thread:

8 Likes
2

Lol you make it sound like I did something, when actually i’m just the “dummy” to test “for dummies” explanations on.

So from that perspective, I might suggest in the readme, a couple of very explicit usage examples of how to make an arbitrary script run. (Which I think I understand, but for the sake of documentation…)

Something like,
Say your script is myScript -

function myScript() {
    // does stuff
}

And you want it to run every time the user visits the dashboard (no matter where they came from)… do this.

Your explanation seems clear enough for a programmer who understands what Turbo even is and what callbacks they should choose, I’m just suggesting making it even easier than that. Which I think is the real beauty of wkof in the first place - it makes making scripts accessible to amateurs, who then end up teaching themselves more as they go, training wheels style. [Ok, me, I’m talking about me :smiley: ]

2 Likes
3

Good point tbh. I’ll see what I can cook up.

2 Likes
4

I’m imagining a higher-order function that just takes as parameters your script main function and a config object, then installs the script to run, hiding all the underlying turbo details.

something like…
function myScript() {…}
wanikaniScriptInstall(myScript, “dashboard”);

Would make the myScript run every time the user visited the dashboard. No having to know the latest tricks with turbo:before-render, etc. Seems (to this amateur) like you’re most of the way there already.

1 Like
5

What you’ve described is actually exactly what’s here already with the callback. Just instead of a single string “dashboard” you would pass in a regular expression for the URL. However, I think having some predefined names like “dashboard”, “lessons”, and “reviews” would be a great idea.

The trouble with just this single function idea is that the library script can’t easily abstract away which of the turbo events your script needs to listen to. The events are named pretty clearly, so I don’t think it’s too complicated to have the script authors need to know which to use. Even novice ones need to learn, not have too much hidden from them, and reading documentation (like turbo’s) is a great start.

1 Like
6

That’s fair. Having figured out what little I know was a learning experience, can’t deny it. :smiley:

7

I totally agree.
Though with that being said, some basic abstraction for the most typical use cases might be warranted, and what I’m thinking doesn’t seem like too much work, really.
If the author uses one of those and it doesn’t work, there’ll still be the rest of the kit available to tinker around with.

2 Likes
8

Okay so that part is basically done (and has been for a while), but while testing around with it before pushing, I noticed some cases where the callback is firing twice during a single load. It’s like there’s no good one size fits all solution.
To try working around that, I experimented with adding a runOnce property when setting the event types (that causes the handler to remove the callback after firing once), which (when set for ‘turbo:load’), in a cursed way, works like 90% of the time.
By that I mean, the following situation and series of navigations causes it to fail to work as desired:

Set up:

let i = 0;
let eventList = ['load', {name: 'turbo:load', runOnce: true},'turbo:before-render'];
let callback = event=>console.debug(`${++i}: ${event?.type}`);
let dashboard = /^https:\/\/www\.wanikani\.com(\/dashboard.*)?\/?$/;
wkof.turbo.on.events(eventList, callback, dashboard)

Navigation:
(new tab) Dashboard → Levels (1) → WaniKani Logo (to Dashboard) → WaniKani Logo (refresh Dashboard)

For some reason, this causes a new page load without before-render, meaning turbo:load is necessary again. But if I remove the runOnce property, it’ll keep calling turbo:load during other situations and thus fire the callback multiple times during a single “page load”.
The weird thing is that further clicks of the logo work properly. It’s only after navigating first that it does this.

Anyway, to meet all my needs, it must:

  1. Run once and only once when the page is first loaded.
  2. Run once and only once when the page is refreshed.
  3. Run once and only once when the page is navigated to from another page.
  4. Run once and only once when the page is navigated to from itself.

But I’m still not quite there (and would definitely appreciate anybody who may lead to a solution for this particular problem).

2 Likes
9

Lol! this is what i was going through yesterday. Thought I had all the cases covered, then just today I caught it one time with two copies of my html on the page. It’s maddening. I think sometimes it fires twice in such rapid succession, asynchronously, that a check that any previous invocation finished fails. That might also mean a “disable” action would have to be really fast, first thing? :man_shrugging:

10

Perhaps this line from the documentation can shed some light on why the event is firing multiple times?

(For turbo:render):

Fires after Turbo renders the page. This event fires twice during an application visit to a cached location: once after rendering the cached version, and again after rendering the fresh version.

Edit: it doesn’t say so for some reason, but there is an implication that if the render event has fired, then the before-render event must have also fired beforehand. And indeed in my testing this appears to be true.

11

I was thinking about dismissing this because I clearly wasn’t listening for that event, but on second thought, I wouldn’t be surprised at this point if that event itself may have side effects for other events.

I’m probably gonna need to sleep on this before I make much progress lol.

12

Ah yeah sorry, please see my edit. Only just thought about it now and we posted at the same time.

1 Like
13

So, judging by what’s happening, it seems like it isn’t possible for the before-render+render series to fire more than twice unless something has really gone wrong, and the load event should only ever fire once. That is, if a client of the library script chooses to listen to “turbo:load” then they won’t have to worry about the double firing; only those trying to fire on “turbo:before-render” and/or “turbo:render” will have that issue.

Also, after doing some searching around, I’ve realized that what we are trying to do is similar in principal to debouncing. Essentially we need to wait and hold off on executing the callback until the last before-render/render event has fired. We can achieve this with setTimeout and clearTimeout and a couple variables outside the handler.

let timeout;
let isCallbackExecuted = false;

window.addEventListener('turbo:before-render', (event) => {
    clearTimeout(timeout);
    timeout = setTimeout(() => {
        if (!isCallbackExecuted) isCallbackExecuted = true;
        // log stuff, call the setup function, etc.
    }, 200);
});

The thing you would then have to figure out is how/when to reset isCallbackExecuted to false again so it can run again when it is supposed to.

14

I mean, it does look like that could work as a workaround, but it is definitely not an elegant solution.
I say that because the user will have to essentially bank on the timing between the multiple events firing not being interrupted by something else, not to mention that deciding upon the timing itself is/would be somewhat arbitrary.
After years of making personal scripts in AutoHotkey, relying on Sleep values is one of those things that ends up being an excessive rabbit hole of debugging.

I think in your example, the isCallbackExecuted is virtually superfluous (at least for the basic implication), since if the listener gets called again before the timeout period has expired, it’ll clear the timeout and start a new waiting period.

To be fair to your idea though, if the script only cares about the first event for any particular load sequence, it could also work in reverse to temporarily stop caring about future events.

For example, something along these lines:

let timeout;
let isCallbackExecuted = false;
let minimumEventTimeoutMs = 200;
let listener = (event) => {
    if (!isCallbackExecuted) {
        isCallbackExecuted = true;
        mainScript();
    }
    clearTimeout(timeout);
    timeout = setTimeout(() => {
        isCallbackExecuted = false;
    }, minimumEventTimeoutMs);
};
function mainScript() {}
window.addEventListener('turbo:load', listener);
window.addEventListener('turbo:before-render', listener);

Essentially, it would function as a fixed interval intermittent schedule of reinforcement (if you’re familiar with psychology).

I might use these ideas at least temporarily though. Perhaps by also adding a configuration variable to decide if it should execute the callback upon the first event or last event and also the timeout length.

15

In this case we don’t need to do anything so complicated, the once property of the options object passed to addEventListener is exactly for this purpose. But in my testing the other events never fired multiple times so that shouldn’t be necessary.

Not superfluous, just related to my comment after that because what it will do is make it so that the callback can never be called again, and that needs to be solved for first. It is essentially the same as the once property, but for the last instance of an event firing, not the first, because it will keep clearing the timeout every instance of the event firing until the flag is true. The chain of events would look like this:

  1. the before-render event fires
  2. we clear any existing timeouts
  3. we set a timeout that will set the flag to true and execute the callback
  4. the before-render event fires again (before the timeout expires*)
  5. we clear the existing timeout so that the callback does not execute
  6. we set a new timeout that will call the callback
  7. the before-render event does not fire again
  8. the timeout finishes and executes the callback

* however, you’re 100% right about the unreliability of timeouts and the need to fiddle around finding a value for the milliseconds that works. But, I think we can take advantage of some intrinsic properties of listening to these particular turbo events, not any old events that can have super arbitrary timing; before-render and render are very closely tied together – one cannot happen without the other and they always come in a specific order and signify specific things: “render has not started yet but is about to” and “render has finished”. There shouldn’t be any other turbo events emitted between these two, and those are the events we care about. These two events are the only ones I’m saying would need this timeout implementation too (saying this since the example you gave seems like you’re trying to apply this timeout setup to all the turbo events, which I don’t recommend as the others don’t have this problem of firing more than once per load due to caching – at least I haven’t observed that on my system).

Personally I haven’t had too much trouble with this by always starting with a value that I know is too high and just lowering it until it’s where I want. I don’t consider this debugging, just tuning.

16

In my test I have found the turbo:render event works fine with setTimeout. A mutation observer is not needed in this scenario.

    function addTurboEventListener(){
        addEventListener("turbo:render", (e) => {
            if (document.querySelector('.dashboard') !== null) {
                // Lets Turbo to complete its work in progress
                setTimeout(delayedExecution, 0); // delay execution to the end of the event queue
            };

            function delayedExecution(){
                let elem = document.getElementById('WkitTopBar');
                if (elem !== null) elem.remove();
                displayItemInspector();
            };
        });
    };

A twice firing occurs only when turbo renders a cache page. We cannot assume whether a given page is cached or not. In other words can cannot know whether the event will be fired once or twice. I solved that by checking whether some change has already be applied and replying them in the second occurrence. This is a solution that is script specific.

3 Likes
17

Ah yes, I forgot to mention that when using the timeout we likely don’t need the observer anymore, thank you.

As for the rest, you’ve reminded me that I was slightly overthinking what would be needed of the timeout. Keeping track of the event loop in my head in regards to microtask queue and timeouts and promises always trips me up. I believe you’re right that the millisecond delay doesn’t matter and that simply using setTimeout will ensure all of the turbo events complete before the timeout callback executes. In that case, this library script would only need to set the callback of the timeout to the callback being specified by the consumer script; however, it would always be up to the consumer to implement their initialization function (or whatever function they pass) in a way that checks for it already having been run in some way and returning early if so. I’m fine with that implementation detail but perhaps others won’t be.

1 Like
18

I have tested the solution posted above with all the following cases

  • Go in the review page and com back to the dashboard.
  • Do a complete review session and let Wanikani come back to the dashboard by itself.
  • Go to the lesson page and come back to the dashboard.
  • Go to an item page and come back to the dashboard
  • Navigation with the navigation arrows between pages
    • Both leaving and going back to the dashboard with forward and backward arrows were tested

Everything works as expected except for one thing. The Failed last Review filter didn’t take into account the failed items when I did the complete review session test. This is a Item Inspector issue. It has nothing to do with turbo events.

I wonder why we would need a library for something so simple as a single call to setTimeout.

1 Like
19

Now that it’s apparent that a single call to setTimeout works very well, I don’t have an answer for this. I will likely continue to implement turbo handling manually myself. But I think the overall discussion is still valuable, and will likely help rfindley when he gets around to modifying open framework to make working with turbo easier as well.

1 Like
20

So, while that could be a great rhetorical question if we ignore the fact that this wasn’t knowledge that we had at the start, there’s still many issues.

If what you’re doing works in your scenario, that’s great, but like you noted, your solution is script specific. Many people are still going to struggle because their situation requires something that doesn’t respond so perfectly, such as not having a reliable way to ignore double events.

And the fact remains that even if it did work fine, it’s all still unintuitive, making it a daunting challenge for new script writers until they decide to search the forums and find this particular discussion, and then they have to extrapolate the knowledge in a way that works for their script.

Edit:
Btw, for the double events thing. Take a look into event.target.hasAttribute('data-turbo-preview'). I believe that is only present during the cached page (source discussion).

1 Like