Wanikani Open Framework [developer thread]

Darn! And here I wanted to be the one saving @prouleau >:C

I know, I’ll throw in an xtra Durtle! :triumph:

2 Likes

@rfindley I have noticed some undocumented features of wkof. I mean these functions.

wkof.wait_state
wkof.set_state

If i understand correctly they are meant to do things like this:

wkof.set_state('taskId', 'Pending');
doSomethingAsynchronous().then(function(){wkof.set_state('taskId', 'Ready')});
//
//  Do stuff that don't need to wait
//
wkof.wait_state('taskId', 'Ready');
//
// Do stuff that needs to wait
//

Do I understand correctly?

I started experimenting with this in Item Inspector. So far it looks good. It appears that I can improve the response time a little with this.

1 Like

Huh, I could have sword that those were documented. I was considering using them for the Review Cache to signal that it was ready to be used, but eventually decided that they weren’t necessary.

Rfindley will correct me if I am wrong, but presumably wait_state returns a promise, so you would need to either use async/await or .then for the stuff that needs to wait. I think for your example you don’t really need to use WKOF at all, just regular promises would do it

1 Like

Yes this is true. It returns a promise. I messed up my example.

And for the documentation, I don’t find these calls here.

1 Like

There is a brief mention of those functions in the Core section of the wkof documentation, but it doesn’t explain their parameters or how they work.

@Kumirei is correct about .then() or async/await.
Other than that, your usage is correct.

Also, be sure to use a task_id that will not overlap with other scripters’ task_ids. For example, add your script name as a prefix:
wkof.set_state('ItemInspector.state', 'Ready');

edit: The main purpose of these functions is to tell other scripts (or tasks within your script) when something has occurred. For example, on wkstats.com, I use it to let certain pages wait until the user’s state is “logged in” before proceeding with API calls.

2 Likes

I have started experimenting with an html type filter. Here is what I have found.

The plan is to implement a better user interface for the Item List filter. I think we should go for a button type because of the limited screen real estate of a filter interface, but let’s do html first.

I want to insert this html. The classes are not there yet. I just check if the html inserts properly.

    let htmlString = '<div>' +
                      '<p>Radicals</p>'+
                      '<input type="text"></input>'+
                      '<p>Kanjis</p>'+
                      '<input type="text"></input>'+
                      '<p>Vocabulary</p>'+
                      '<input type="text"></input>'+
                     '</div>';

I have modified to filter loading code to handle html like this:

                    case 'html':
                        flt_content[src_name+'_flt_'+flt_name] = {
                            type:flt.type,
                            label:flt.label,
                            html:flt.html,
                            default:flt.default,
		                 	hover_tip:flt.hover_tip,
                        }
                        break;

According to the doc default and hover_tip are not going to be used, but I included them anyway. The result is this.

html

Upon inspecting the elements I have found that:

  • The row is not wrapped into a <div class="row">
  • There is no checkbox and no div to include one.
  • The html is not wrapped into a <div class="right">

These elements are present in all the good looking rows, so I guess they must be included for html as well.

Then there is this: ( from the wkof doc)

The html component does not store a value. However, you can attach event handlers to the html via the global pre_open callback.

In the context of a filter I am not setting up the dialog. I don’t have access to the pre_open callback. I don’t see how I can attach event handlers.

And now this:

I have looked at the code. This is not going to work because there is no path associated with the individual elements. setting_changed() stores them all at the same path which is associated to the whole html type.

Some suggestions

I think we can avoid the complexity of associating paths to an unknown html structure by storing an object in the lone path provided by the framework. The html callback and event handlers will have the responsibility to map this object to the individual elements.

I suggest that an html configuration should be like:

html_id: {
    type: 'html',
    label:          // (optional) A string label that appears to the left of (or above) the inline html.
    html:           // An html string to be inserted inline.
    path:           // The path where the object value will be stored
    default:        // A default object value
    pre_open:       // A callback that will be called prior to the opening of the dialog
    hover_tip:      // A hover tip that will be applied to the label but not to the html.
}

The signature of pre_open should be:

pre_open(root_elem, object, on_change)

root_elem : The DOM element where the html is located.
object: The object stored at the path when pre_open is called.
on_change: A callback that will inform the framework of a new object value

The signature of on_change should be:

on_change(object)

object is the new value to be stored in the settings.

I think this solution will work for a button as well except that instead of ‘pre_open’ it will be ‘on_click’ and the root_elem.parameter is not required for a button.

You have more experience than me with this sort of things. Do you see flaws or ways to improve on this?

Edit: I made another test for html.

I asked for this string.

    let htmlString2 = '<div class="row">' +
                          '<div class="enable">' +
                              '<input type="checkbox">' +
                          '</div>' +
                          '<div class="left">' +
                              '<label for="for="Item_Inspector_wk_items_flt_listed_items"">' +
                                   'Listed Items' +
                              '</label>' +
                          '</div>' +
                          '<div class="right">' +
                              '<p>Radicals</p>'+
                              '<input type="text"></input>'+
                              '<p>Kanjis</p>'+
                              '<input type="text"></input>'+
                              '<p>Vocabulary</p>'+
                              '<input type="text"></input>'+
                          '</div>'
                      '</div>';

In other words I tried to do manually what is expected from the framework, and I removed the label from the registry function.

	function registerItemListFilter() {
		wkof.ItemData.registry.sources.wk_items.filters[itemListFilterName] = {
			type: 'html',
            html: htmlString2,
			//label: 'Listed Items',
			default: false,
			filter_func: itemListFilter,
			set_options: function(options) { return; },
			hover_tip: itemListdHover_tip,
		};
	}

The result is this.

html

This is an improvement. The checkbox and the <div class=""row> container were added on top of those I provided. And I get the html properly styled. So it seems that requesting a label while registering is buggy. We can workaround the bug by providing the label manually.

To test that I tried this html. I don’t request a label and I supply one manually. My expectation is that if I don’t provide a checkbox and a class="row" container the framework will provide one and there will be no duplication.

    let htmlString2 = '<div class="left">' +
                          '<label for="for="Item_Inspector_wk_items_flt_listed_items"">' +
                               'Listed Items' +
                          '</label>' +
                      '</div>' +
                      '<div class="right">' +
                          '<p>Radicals</p>'+
                          '<input type="text"></input>'+
                          '<p>Kanjis</p>'+
                          '<input type="text"></input>'+
                          '<p>Vocabulary</p>'+
                          '<input type="text"></input>'+
                      '</div>';

The result is this.

html

Now there is no checkbox and no <div class="row"> container. The bug is no so simple as I thought.

It would be good to have the label on top of the html and have the html use the full width of the filter panel. This would be a much better use of the screen estate. This filter would need it. Is there any provision in the framework for this? Or do I have to cook up some css?

I did more tests hoping to find a html that works. I failed. But I have to report that this one gives me a crash.

    let htmlString2 = '<div class="row">'+
                           '<div class="left">' +
                              '<label for="for="Item_Inspector_wk_items_flt_listed_items">' +
                                   'Listed Items' +
                              '</label>' +
                          '</div>' +
                          '<div class="right">' +
                              '<p>Radicals</p>'+
                              '<input type="text"></input>'+
                              '<p>Kanjis</p>'+
                              '<input type="text"></input>'+
                              '<p>Vocabulary</p>'+
                              '<input type="text"></input>'+
                           '</div>' +
                      '</div>';

html

Keeping you posted. I have managed to insert html properly. I am working out the css that goes with this. For now it looks like this.

html

I also managed to get it full line.

html

Works fine in a filter with checkbox too.

html

But in a filter it looks better full line.

html

Once I am happy with the css I will work out the callbacks.

Bug report: the framework uses the same buggy regular expression to handle white spaces as the item list filter. I ran into an obscure bug because I put white spaces around my commas in wkof.include and wkof.ready calls. I can work around the bug by removing the spaces but it would be nice if a working regular expression is used.

The offending line of code:

function split_list(str) {return str.replace(/^\s+|\s*(,)\s*|\s+$/g, '$1').split(',').filter(function(name) {return (name.length > 0);});}

I’m back from vacation, so I took a look at this.

Can you give me an example string that isn’t working? The regex is designed to accommodate spaces around commas, and I’m not seeing any functional difference between the existing code and your suggested replacement from the Item List Filter thread.

replace(/^\s+|\s+$/g, '') is equivalent to trim().

1 Like

I tried to reproduce the problem and you are right. There is no functional difference between the two. It seems I have misdiagnosed the problem. Sorry about that.

But I have noticed that when I had the problem with the Item List filter I entered Japanese characters using an IME. The Japanese punctuation and white spaces were used. This might be the issue. Here are screenshots of the problem.

sample

This selects this

result

If I replace the Japanese punctuation with latin characters the problem goes away.

Ahh, that makes sense. I’ll add the japanese comma and whitespace to the filter. (Will have to wait for some free time again, though.)

3 Likes

@prouleau, this is what I’m planning to update to. You are welcome to test it and let me know if you find any issues.

function split_list(str) {return str.replace(/、/g,',').replace(/[\s ]+/g,' ').trim().replace(/ *, */g, ',').split(',').filter(function(name) {return (name.length > 0);});}
1 Like

With this configuration:

Settings

I get this result.

result

In other words, if I use Japanese commas I still get garbled results. But latin commas are OK.

It is the same in both Item Inspector and Self Study Quiz. This is not a script issue.

I have verified that the code you propose for split_list produces the correct string. This proposed code is OK to use. The problem is elsewhere.

I have verified that the .kanji_list.indexOf(item.data.slug) call used in the kanji filter locates the slug at the correct index.

I have verified that the slug contains the correct character.

I have verified that you use the correct unicode characters.for Japanese in split_list(str).

I have placed some console.log() command in the filter functions to get more information on what is going on. There is no console output as if the filter was never called. But this is absurd. The filter is obviously called because I see the items being filtered and all other filters are disabled.

I have copied your code in one of my test scripts where I know console output works. Guess what? I get console output and I can’t reproduce the problem there. Using Japanese punctuation works correctly in the test program.

So I investigated the greasyfork directives in your script to see whether the explanation is there. I removed the @run-at document-end directive. This is the culprit. Without this directive I get console output and the filter works fine.

You have to replace split_list() in the “Item List Filter” script and wkof’s ItemData module. (And also in wkof’s Core module when using Japanese commas or spaces in wkof.ready() ). And then you also need to clear or disable wkof’s cache to make sure you’re loading the updated version of the ItemData module.

I was able to get it working. (I’ll upload when I can. Having some problems with my editor at the moment).

This is obvious. I did that and still got the problem. I traced it to the @run-at document-end directive as I explained. Your code is fine but it doesn’t work for me until this directive is removed.

Edit: I just tested it on Firefox. This is a browser specific thing. The problem occurs on Chrome, not on Firefox.

I don’t think it’s a browser issue, and at most it’s maybe tangentially related to @run-at. I don’t doubt your results, but there’s no evident logic to them yet.

It works fine on my Chrome without modifying @run-at, as I would expect. I was getting bad results before changing split_list() in:

  • wkof ItemData module
  • wkof Core module
  • Item List Filter script

I modified those, then cleared wkof.file_cache.

After that, with the filters configured as you showed, all of the expected items (and nothing else) were showing up in the quiz.

1 Like

I am unable to reproduce your result. I changed split_list() in

  • wkof ItemData module
  • wkof Core module
  • Item List Filter script

I cleared wkof.file_cache. I did the whole thing twice to make sure there was no mistake. I still have the problem and there is no console.log() output from the traces I have put in the filters. But as soon as I removed the @run-at directive the problem disappeared and the console.log() output appeared.

I don’t understand the logic behind your approach. If the wkof modules and the cache were causing the interference, why would removing the directive work? It is the same wkof and the same cache running. Also why would the framework have an impact on console.log() output?

Besides I was able to get the filter running without changing the modules and clearing the cache when I removed the directive. I think this shows there is no interference from wkof.

An additional point: the current wkof (with the old split_list()) works in Firefox for me. There is no need to upgrade wkof on Firefox.

I don’t doubt you got the result you stated but it doesn’t work like this for me.

Clearing the cache is a drastic action. I lost all my settings. Are you sure you want to roll out a solution like this?

Edit: I think we are facing a bug involving the directive when running on Chrome. This would explain the inconsistent behavior and why the problem doesn’t appear on Firefox. There is no logic in bugs.

That’s where we differ. There is always logic in bugs, just not always evident until you’ve found the root cause.

Your screen-shots show exactly the result I would expect if the split_list() is not processing Japanese commas, and is exactly what I got before updating it (though I am not using your script to get results, I am using Self-Study Quiz). ground and gambler radicals are properly recognized due to latin commas, and ookii has a latin comma separating it from the rest of the line, including the immediately following Japanese comma. So, logically, split_list() isn’t being update somewhere. It exists in the three places I stated. Do you have another copy in your script? I haven’t installed it.

So, I fixed split_list()in those 3 locations, cleared cache to ensure the new version loads (which will not be necessary after I change the version number while publishing), and Self-Study Quiz gets the right results: 12 hits in the quiz… two each for kanji and vocab (reading and meaning) and one each for radicals.

So, from my perspective, the issue is resolved for wkof and self-study quiz. Any remaining issues are likely in your script, which I don’t have installed. Maybe you do have a timing issue in your script, but until you understand the root cause, blindly changing @run-at isn’t the step I would take. Try looking at the order you are intending to do things (e.g. load filters into the registry, then execute code based on them, etc) and think about what mechanisms are ensuring that things are really happening in that order.

2 Likes

I must disagree. My reasons:

  • wkof with the old split_list() works in firefox. This is not an issue of wkof not being up-to-date. Please test it and see for yourself.
  • The console.log() output appears and disappears depending on whether the @run-at directive is present. Something other than split_list() is happening.
  • My script loads filters using code I have borrowed from Self Study Quiz. There is no special check for the order of handling filters in this code. It just iterates over the registry. So if there is an issue with that we both have it.
  • I don’t use split_list() in my script.

Note: The console.log() out disappearance occurs when the console.log() command is located in the filtering function. It doesn’t happen when outside this function.

Is there a way to hide the progress popups that wkof automatically creates for certain api requests?