Wanikani Open Framework [developer thread]

rfindley · September 30, 2020, 4:27pm

And the return value is ‘true’ if the item should be kept, or ‘false’ if it should be removed.

[edit: Forgot to mention… No, the filter interface is undocumented.]

prouleau · September 30, 2020, 5:34pm

I also have to figure how to handle the registry and the settings interface but I have found it in @seanblue 's code and in the Self Study Quiz code. I am good with this.

prouleau · October 4, 2020, 2:57am

@rfindley I want to implement some datetime filters. The problem is datetime is not supported as a native data type by WKOF. The implications:

Datetime are stored in the settings as text strings and not binary count of milliseconds.
Datetime format must be validated by the validate callback, but the user interface generation for filters in Self Study Quiz does not support these callbacks.
The built-in regex validation is not an option for the same reason: no support in Self Study Quiz.
It follows that the filter will be provided its datetime parameter in the form of an unvalidated text string.
The filter will have to parse the datetime at every invocation with no sensible mean to report datetime format errors. The best it can do is to log something at the console. Given how many times a filter is called this will cause a lot of console spam.

I don’t think I can come up with a sensible filter with these limitations. Anyone sees a way around it? Am I missing something?

I have looked at the button and html options provided by WKOF. Self Study Quiz does not support html and there is no value attached to a button. I don’t see how to turn these options into a filter.

rfindley · October 4, 2020, 10:52am

Sorry, I can’t do an in-depth look at the technical details at the moment, but:

Have you looked at the ‘filter_value_map’ parameter of the filter? It lets you add a function to pre-parse the filter value to another format before iteratively passing the new format to the filter function. Example:

filter_value_map: function(datestring) {
  return datestring.get_time();
}

Also, in your copy of Self-Study, try adding support for validation and let me know if you get it working, and I’ll merge it. Example:

case 'text':
case 'number':
case 'input':
  flt_content[src_name+'_flt_'+flt_name] = {
    type:flt.type,
    label:flt.label,
    validate: flt.validate,   //Add validation func from filter
    placeholder:flt.placeholder,
    default:flt.default,
    path:'@ipresets[@active_ipreset].content["'+src_name+'"].filters["'+flt_name+'"].value',
    hover_tip:flt.hover_tip
  }
  break;

prouleau · October 4, 2020, 11:27am

Thanks for the advice. I will look into this.

For the validate parameter there is one issue I can foresee. I need to make a function declared in my script available in the name space of Self Study Quiz. I risk getting some reference error or something. What is a good procedure for making a function available in both namespaces?

I didn’t know there is a ‘filter_value_map’ parameter. It is not used in @seanblue 's code I used as a reference. The lack of documentation hurt here.

Can you provide a complete list of available filter parameters?

Kumirei · October 4, 2020, 11:40am

I don’t know if this is good practice or not, but what I have been doing is exposing my functions as window.<script name>.<function>

prouleau · October 4, 2020, 11:45am

Thanks. If nobody comes up with something better I will use this.

rfindley · October 4, 2020, 12:19pm

It’s not a problem. When you set filter_value_map, the actual function is being passed (via internal js pointer), so there is no namespace to be concerned about. Even anonymous functions are fine.

Sorry about the documentation. The lack of docs is due to the filters essentially being an extension of Self-Study Quiz rather than a feature of wkof itself. For a long time, only @seanblue and I were using them, and we added features as the need for them became apparent.

The list of parameters actually used by wkof are:

type - Field type (“text”, “number”, “dropdown”, etc)
label - Appears next to the field in the dialog
placeholder - Value of ‘placeholder’ attribute on the field.
default - Default field value
filter_value_map - Function that receives the filter value set by the user, and returns a transformed value to be passed to the filter_func function.
filter_func - Function that is passed each WK item and a filter value, and returns true if item is to be kept (i.e. not filtered out).
hover_tip - Text that appears if you hover over the field.

Technically, you can add any parameter to a filter, and it’s up to the scripts that use filters to know what to do with them.

prouleau · October 4, 2020, 12:46pm

Thanks a lot. You are a great help.

Now there are three of us. Item Inspector is replicating Self Study code to use filters to build its tables. Putting filters in the framework even though Self Study was the lone application was a great foresight.

I have a need to run filters over a set array of items instead of fetching them from the framework. What is a good way to go about this? Is there something I can call? Or do I need to iterate over the registry and manually call the filters? Can you point me to some sample code?

rfindley · October 4, 2020, 12:58pm

The only thing wkof.ItemData.get_items() does is pull the full items list from cache and iterate over the specified filters. So, if you want to do a subset of items, just create a filter that extracts the subset from the full set.

Kumirei · October 4, 2020, 12:58pm

I made some filters (JLPT, Joyo, Frequency, Visually similar, Subject ID) in Feb 2019, so you’re the fourth

prouleau · October 4, 2020, 1:41pm

Thanks for pointing this out. I will look at how to integrate these filters into Item Inspector.

Are JLPT, Joyo and Frequency the same filters than those I see in Self Study by default? I think they are in the default registry because I see them in Item Inspector too.

Visually Similar is interesting. I now only display the Wanikani visually similar available from the API. I should use this github page information somehow, not just for filtering.

I don’t find Subject ID. Where can I get it?

prouleau · October 4, 2020, 1:46pm

This is another no user interface filter. I don’t want this to show up in the settings. I want to control whether this filter is on or off programmatically depending on whether or not I want a subset.

Kumirei · October 4, 2020, 1:52pm

I’m not aware of any such default filters. Are you sure you don’t just have the filter script installed?

I actually removed that one because I decided it was implemented pretty poorly. For my purposes indexing the WK items by subject ID using wkof.ItemData.get_index(items, 'subject_id') and then accessing them worked just as well

prouleau · October 4, 2020, 2:06pm

I am sure. See for yourself. This is my script list.

Lol I was doing the same for a while. But I noticed that most of the time I know the item type (Rad, Kan, Voc). So I now manually build three indexes, one for each item type, and search the right index. When I don’t know the item type I search the three indexes in sequence. Searching smaller indexes led to a noticeable speed improvement and made Item Inspector more useable.

Kumirei · October 4, 2020, 2:07pm

Oh, you know what, I probably @require it in Progress Percentages

Kumirei · October 4, 2020, 2:10pm

Really? It’s just 9000 items and key/value maps are pretty efficient In the Heatmap I don’t know anything but the subject ID for reviews, so I can’t do that anyway

prouleau · October 4, 2020, 2:13pm

Yes you do. I disabled Progress percentages and the filters disappeared so this is where they are coming from. This is good to know. I must notify the Item Inspector users.

prouleau · October 4, 2020, 2:18pm

I didn’t take a measurement but subjectively the speed improvement was noticeable when I made this change.

Kumirei · October 4, 2020, 2:44pm

Or you could also @require it if you want it available to everyone