Wanikani Open Framework [developer thread]

I’m tinkering again…
What is your contribution policy on this project?

Nice! What do you have in mind?

I haven’t really put together a contribution policy yet, but…

Essentially, I’ll be happy to accept contributions that stick to the main goals of the project, plus a few design guidelines, and a few caveats.

Main goals:

  • Things added to the framework should be of general interest so that it doesn’t grow into a large collection of one-off stuff that may be difficult to maintain when Wanikani changes. If something is useful, but isn’t both ‘generic’ and ‘general interest’, consider whether it may be worth building a separate library that can be installed alongside the framework (such as seanblue’s Additional Filters script, which is ‘general interest’, but has some specificity that makes it not quite ‘generic’, such as its particular definition of ‘leeches’).

  • Stick to 80/20 design: The API should aim to be simple for 80% of use-cases, but should leave room for the other 20% to customize. In other words, using the framework should be relatively painless for newbie scripters, but we still want it to be flexible enough that experts won’t feel locked into a one-size-fits-all API.

Design guidelines:

  • Always think about how your code might interact with multiple client scripts. This includes race conditions, script load order, manipulation of the DOM, use of variables in global space, etc.

  • Visual elements should be as independent of WK as practical. For example, the Settings and Progress dialogs are placed in a top-level DOM container created by the framework in order to avoid dependencies on the Wanikani layout and CSS. If Wanikani did a total site makeover, I think those modules would still function the same. There are some exceptions, of course. The Menu module depends on Wanikani’s interface. But it’s also light-weight, so it will be easy to maintain.

  • If creating a new module, be sure to follow the general design of the existing modules:

    • Wrap everything in an anonymous function to prevent accidental overlap in the global space.
    • Make a ‘Published Interface’ section at the top of the code that gives a good idea of what the module does.
    • Use heirarchy to keep things organized, especially in the wkof object.
    • Modules must set a state variable when they are ready to be used. See the notify_ready() function in Apiv2.js for example.

Caveats:

  • I can’t (and don’t want to) own the framework forever, but I do want to curate it somewhat for a while so it stays clean and true to its purpose (e.g. “Too many cooks spoil the broth”). My hope is that, if people want to continue adding to the framework, someone will show enough interest to take over curation. Because, as things are right now, if the framework breaks, WK is going to suddenly be a very different place for a lot of users!
  • I don’t have a lot of time to spend on integrating new stuff. So, since it’s a lot faster to review code than to write it, I would encourage keeping in frequent contact so I can help guide the direction of anything intended for integration. I’m more than happy to answer questions and give suggestions, so don’t hesitate to ask :slightly_smiling_face:.

I can give out my email address if anyone wants to discuss anything offline. But if the discussion is useful to the developer community, just post in this thread.

1 Like

Speaking of modifications to the Open Framework… With only about 30min of work, I managed to get the framework working in a staging area for the new wkstats.com site. That’s really exciting because it means it’s going to be easy for other developers already familiar with the framework to be able to contribute to the stats site. My hope is that scripters can develop their ideas in userscript form on top of the stats site, and then those ideas can be quickly integrated into the site.

I’ll post more detail as I make progress.

3 Likes

I am in awe of your code and trying not to feel stupid with how narrow my focus has become from my job. It’s hard for me to look at something like this and not think that the author just sat down and coded the whole thing over a weekend or afternoon.

I’ve made a couple pull requests but sometimes think the problems I solve will either cause more problems, or the solutions were already considered and discarded for some reason.

1 Like

Heh, nothing to worry about. Just put some ideas out there and we’ll talk through how they might be implemented. You learn (or relearn) by doing.

1 Like

@seanblue @normful @Kumirei @DaisukeJigen @valeth @Ethan

I’ve updated the recommended code for checking if the Open Framework is installed. You may want to consider updating your scripts accordingly.

The intent is to give the user a choice about whether to redirect to the Install page if the Open Framework is missing or outdated.

The new recommendation is [here].

Thanks to @seanblue for recommending the use of confirm() instead of alert().

3 Likes

Thanks for the notification (and all your work on this framework!). Updated!

@rfindley I wanted to get your opinion on how to do something. I might want to reuse one of the filters that I wrote for the Additional Filters script for another purpose. However, the filter itself may not be registered with WKOF if the user has turned it off in the Additional Filter script’s settings. So here are my questions.

  1. My filter has a parameter with a default value. When it’s used in Self Study, it can be on or off and have different settings depending on the item set being used. How can I have a unique setting for overriding the default for my script, just like the Self Study script does for each item filter?
  2. Related to the previous question, how can I use Open Framework to just apply this one filter and none of the others registered with the framework?
  3. Any suggestions for what to do if the user has disabled the requisite filter in the Additional Filters script? I could just force it to be included if the new script is enabled, but I’m not sure if that’s the best approach. Is there any way for my new script to use it without forcing it to show up in Self Study if the user turned it off in the Additional Filters script? (I’m kind of regretting having added the global setting for not registering the filters hehe.)

If it makes sense, feel free to just point me to specific code in the Open Framework or Self Study scripts.

You’re wanting to use this filter in something other than Self-Study, right?

The settings in the Self-Study dialog are stored purely inside of wkof.settings.ss_quiz, and are only used by Self-Study. If you want to keep separate settings (in addition to the default), you can store them in your own script’s settings, and just pass those settings to wkof.ItemData.get_items(). That’s all Self-Study is really doing… a fancy dialog box built dynamically based on the description in the registry, but ultimately just passing the user’s chosen settings to get_items() like this:

wkof.ItemData.get_items({
    wk_items: {
        filters: [
            some_filter: {value: some_filter_value},
            another_filter: {value: another_filter_value},
        ]
    }
})
.then(do_something);

If you want to use a filter without registering it, just apply it manually to the items returned by get_items():

wkof.ItemData.get_items()
.then(additional_filter)
.then(do_something);

function additional_filter(items) {
    // We'll use the Array.prototype.filter() function to iterate through each item.
    // 'your_filter_function()' should return true for each item you want to retain.
    return items.filter(your_filter_function);
}

function your_filter_function(item) {
    // return 'true' if you want to retain the item in the dataset
    // Example:
    var some_date = wkof.settings.my_script.somedate;
    return (item.data.unlocked_at > some_date);
}

function do_something(items) {
    // TODO: Do something with the fully filtered item set.
}

In summary, filters don’t have to be registered. The registry just helps scripts find new filters dynamically, and tells the script any info it needs to know to be able to dynamically create a user interface for such filter.

Are you asking how to set up your Settings dialog in a similar way to Self-Study? Or just how to keep separate settings?

No need for a settings dialog like in Self Study. I’d just put it in the main menu with other script settings.

1 Like

How are these two approaches different? Is it just that the first requires the filters to be registered (if that’s even right)? Anything else?

Yeah, mainly just registered vs non-registered. Internally, the Framework is doing pretty much exactly what the second one is doing, but it’s slightly more efficient since it applies all filters to each item, so it doesn’t have to iterate over all of the items multiple times. But the speed difference it negligible for only 8804 items.

The only other difference is:

  • Registered filter functions are passed two values: the filter value, and the item.
  • The way I set up the non-registered function in my example, it is only passed the item. But you could pass the filter value too, if you want.
1 Like

@rfindley Is there a built in way to get a specific set of items? It would be a shame to have to run a custom filter with each request, especially since I’d have to search through every item to find the right ones.

For testing purposes, I built a ‘slug’ filter that takes a comma-separated string containing the list of slugs. I could post the code later if you want.

For every query, the Framework always grabs the whole list of items from indexedDb (if it’s not already in memory), and just runs the Array filter() function on all items. So, writing a Framework filter is essentially the same as just taking the full items array and just running the array filter() function on it.

@seanblue,
The Settings module now remembers the selected tabs for settings dialogs.

One caveat, though: It remembers if you close the dialog with Save, but not with Cancel, because the state is stored inside the client script’s settings (wkof.settings[script_id]).

1 Like

I’m just writing my first little script using the wkof and it so nice and helpful and does exactly what it is supposed to do according to the documentation. Thanks so much!

And happy cake day to you, @rfindley. 4 years of making WaniKani a better place :heart:

3 Likes

New script!

Looks nice!
Added to the top-post.

1 Like

Thanks for making the WKOF. Writing this script was so much easier with it.

1 Like