Wanikani Open Framework [developer thread]

Thank you! That does seem to confirm then that the heatmap will correctly have access to the review session IDs for Custom SRS, so it’s just a question of how I implement the new source’s fetcher function.

I’m hoping Kumirei might be able to tell me how the heatmap calls it - that way I can know what exactly I need to return / what parameters I need to pay attention to without having to go through all the heatmap code :sweat_smile: Thanks for pointing out get_items() - I hadn’t seen that function.

1 Like

Heatmap doesn’t define a new source at all, so it won’t be able to help you with the fetcher. Review Cache, which it uses, also just uses a combination of the builtin Open Framework fetching and manually keeping the reviews in the file cache.

The fetcher really just needs to be a wrapper to whatever function needs to be called to actually fetch the data you’re providing. This wrapper would do things based on the passed in config options, then when it is done open framework will apply the passed in filters if specified (which would mean you’d need to also register new filters in open framework if the default ones or the ones provided by the Additional Filters script or Reorder Omega or the Item List Filters script don’t suffice). If you aren’t providing any data to be fetched, you don’t need to add a source.

To be honest, I don’t think I’ve ever seen a script actually register a new source with Open Framework. Not even the insanely dense Item Inspector (another one that adds lots of filters; but no new source).

1 Like

Hmm, it seems I’m completely misunderstanding how the Heatmap uses WKOF then. My assumption was that it stores the review data with the Review Cache, and then uses WKOF to get info about those items so that it can show them in the heatmap popups. If that were the case, I’d need to know what info heatmap is asking the WKOF get_items() function for so that my custom fetcher function could return the same.

I’m almost starting to wonder though if it would be easier to just skip WKOF and have a globally available function (e.g. custom_srs.get_items() or something) which the heatmap could then call when dealing with items with negative ID… Any thoughts @Kumirei ? Would that work just as easily for you / would you be able to make use of that? It of course wouldn’t register it for other WKOF-using scripts to be able to get the data - but it seems like even if I did register a new source it could be complicated to make sure I’m returning all appropriate data etc. for any other scripts that might use it / would they even automatically get that data without having to make code changes… :thinking:

Thanks for your help on this!

1 Like

Oh sorry, no, I was just misunderstanding what you wanted to do because you were talking about Open Framework’s source registry stuff.

That is indeed what Heatmap does. The relevant function is async function update_popper(event, type, title, info, minimap_data, burns, time) in the source.

You would essentially need to structure the items you are storing the same as items returned by Wanikani’s API I think. Then one could use the same filters for your fetcher. But the options config would be an issue, because you don’t have all the same endpoints as Wanikani (indeed if you’re not making a whole API you don’t have any endpoints really). Perhaps it would be omitted and you just return all stored items when fetched (which would then be filtered).

2 Likes

They wouldn’t automatically get the data no. They would have to fetch it, just like with wanikani data, by supplying a config with your source name as a key. It would just be the case that the source becomes available to any script once it is registered.

2 Likes

If I recall correctly the traditional radical feature of Item Inspector is implemented as a new source. I am not sure about Item Inspector main file but the optional filters here defo register a new source…

2 Likes

Ah, I didn’t think to check if the filters themselves were also registering new sources, thank you. Item Inspector’s main file is all I checked even though I was pretty sure the traditional radicals had to be a new source.

1 Like

@Hubbit200 I am calling

await wkof.ItemData.get_index(await wkof.ItemData.get_items('include_hidden'), 'subject_id')

I don’t mind either including your source here, or separately calling a global function custom_srs.get_items() like you suggested. Should be a similar integration for me regardless

1 Like

So here’s a riddle regarding a WKOF interaction I really hope someone might be able to help with…

Before continuing with troubleshooting a website account-access issue, my bank insisted I clear all my cookies and site data and disable all extensions this morning and I complied with this ridiculous request just to show them it wasn’t the issue. I didn’t want to — I’d already tried Chrome and Opera with the same results, but they insisted Safari was the only one they’d troubleshoot me with. (And surprise! It wasn’t my issue, it was on their end.)

But now, upon re-enabling userscripts, The Show Specific SRS script is the only one I use on WaniKani that seems to activate, no matter how many times and ways I do the reload dance, restarting the browser and machine, etc.

The ones I can’t get working that I consider irreplaceable are Jitai, Double-Check, and Ultimate Timeline, (and besides of course WKOF itself, I’ve disabled all others). For these past weeks since the redesign, I have always have to do the reload-some-indeterminate-number-of-times thing to get them to work, but they were all working for my review session this morning before the bank call.

Scripts on other sites seem totally unaffected (but I only use 3 scripts on any other sites anyway).

Besides the obvious, namely that those three use WKOF (the reason I posted this here, since if not here the only other thing I could imagine was to create a new forum topic) — can anyone think of an actionable difference that might help me to find a workaround?

Losing Double-Check, especially, is just soul-sucking.

I know getting WKOF updated is a major effort, but the reload trick worked for me up till now… I’m wondering if it’s maybe the case that by disabling not just the scripts but the extension, I lost persistent state that was the only thing allowing it to work after a reload, or something else that I might be able to work around?

Oh also, in case this is helpful in figuring out a workaround, some new behavior as of this afternoon: the first time I attempt to start a Review after a new login to WaniKani in the browser, it never loads.

I just stare at the Dashboard screen with a very slowly moving loading bar that eventually fills and goes away (after about 90 seconds!) leaving a search box with the https://www.wanikani.com/subjects/review address, but never¹ actually switching to the Review.

Instead, I have to interrupt the loading with ⌘-R to reload the Review — which gets it started, but without the WKOF-dependent scripts.


¹ With “never” meaning “up to about 5 minutes, which is the longest I waited”.

Uh… I guess this was a heisenbug. Immediately after having posted these last two comments, these scripts just suddenly started working.

Still only after reloading, but working. I have no explanation. ¯_(ツ)_/¯

Probably started working due to all of the item data finishing loading while on the Dashboard.

1 Like

Indeed—the time I spent writing comments on here probably was the longest period I left the Dashboard tab active without trying a review.

Some extension I used to have enabled would occasionally pop up a JavaScript faux-window with a progress bar saying “loading WaniKani data” or something like that. I haven’t seen it recently so it might be one I disabled. But I do recall now that if it was up when I tried to start a Review, it would also result in the Review never actually loading, just like I was experiencing above.

Ah well, in any case, seems to be working for me now, apologies for the catastrophizing!

And I just cleared my wanikani.com site data to try to troubleshoot why the tab was logging me out of WK several times a day, and once again the scripts aren’t fully loading. We’ll see if an extended period sitting on the Dashboard page first fixes things again…

Interestingly, in Reviews the gear icon (which for me leads to settings for both Double-Check and Jitai) and lightning icon (from Double-Check) initially doesn’t appear, then usually after one reload the gear icon appears but only for Jitai (and the lightning icon does not), then on successive reloads the lightning bolt appears (and the gear icon leads to both Jitai and Double-Check settings) and remain across subsequent reloads, even while neither script actually works.

While I’m not sure about the script issues, the login issue is due to this:

1 Like

Thanks, that must have been it. I wish I had caught that or I wouldn’t have cleared my site data. I left the Dashboard tab open and active (after a reload so Ultimate Timeline had loaded and was visible) for an hour and neither Jitai nor Double-Check work no matter how many times I reload (though, as I mentioned, the gear and lightning-bolt do appear after reload).

Any other suggestions of ways to kick things back into place?

Well, I’m planning to fix Double-Check and Ultimate Timeline soon. I finally finished my huge work project, but now I have a handful of small things to catch up on that I’ve been neglecting during that project. As long as the script changes don’t turn into a bigger task than I think it will be, I should have it done this week.

2 Likes

Ah, okay. I find lack of Double-Check even more disruptive than most I think because I use it almost exclusively to mark items wrong when I feel like I don’t actually have them (I have the wrong sense of a meaning, I had to think about mnemonics for something I was about to burn, I was wrong on transitivity but WK let the answer pass, etc.).

You can work around not being able to undo wrong answers by reloading, but you have no such option when you get something marked correct you wish hadn’t been.

And I have found a workaround: I’d used the system Settings app to clear wanikani.com browser data to inadvertently cause the issue.

So after failing to get these extensions to function and writing the above comments, I used Safari’s built-in website content settings dialog (available using the icon on the left of the search/URL box) to clear all website data. At that point, a reload of the Dashboard caused the “downloading item data” box to pop up.

When I then started a Review, all the Double-Check and Jitai settings had been cleared (so Jitai had no fonts besides the default to choose, and Double-Check was set to not do anything) so I re-set them to what I’d had before and now they’re working again.

A couple of my Jitai fonts are missing now and I have no idea how to get them back, but otherwise things seem to be working okay.

I’ve updated Open Framework to improve support for Turbo navigation.

Wkof now contains:

wkof.on_pageload(urls, handler)

Example usage:

wkof.on_pageload([
    '/',
    '/dashboard',
    '/recent-mistakes/*/quiz'  // Supports wildcard
], script_startup);

let first_load = true;
function script_startup(url) {
    // Turbo doesn't modify the contents of <head> when navigating, so
    // if you install CSS into <head>, you only need to install it once.
    if (first_load) {
        install_css();
        first_load = false;
    }

    // Your normal script setup goes here:
    setup_everything_else();
}

I’ve also updated Double-Check to make use of this, so you shouldn’t need to ‘refresh’ every time you go to Reviews now. (Unless you have other scripts that still need the refresh.)

Note: Any scripts that used the undocumented wkof.on_page_event() will need to be updated to use wkof.on_pageload() instead.

3 Likes