Good to know. I will adjust to the changes on my end.
1 Like
Just to be clear, you want me to add the set_state call in my Show Number of Learned Kanji script?
Or were you talking about my planned WK kanji search userscript using item filters?
If so, you probably want me to call wkof.set_state('yourScriptId', 'ready') after i’ve used wkof.ItemData.get_items(config)? Because i’m not using items.filter, the function using that is unused. Or where am I registering filters in myscript?
Here’s the relevant (?) excerpt of my script:
wkof.ready('ItemData').then(getItems).then(mapItemsToSrs).then(addNumberOfLearnedItemsSection);
function getItems(items) {
return wkof.ItemData.get_items(config);
//return wkof.ItemData.get_items(config).then(filterToActiveAssignments);
}
1 Like
You don’t need to do this because you are not creating a new filter as I thought you would. Therefore you are not registering a new filter. You just use existing filters. This is good enough for your purpose. I don’t need to synch with your work in this case.
1 Like
Done.
wkof.set_state('item_list_filter', 'ready');
1 Like
Thanks you very much.
1 Like
Will you do Part Of Speech as well?
Yeah. Seems like just some thread get wider. I don’t have time to look at it any more today, though
1 Like
I forgot about that one. I’ve never actually used any of my filters, other than the built-in ones.
Anyway, it’s added now.
wkof.set_state('pos_filter', 'ready');
2 Likes
Thanks again.
Item Inspector users will be ale to turn on all available filters with settings. They will be off by default so Self Study Quiz is not overwhelmed with a huge list of filters. This way there will be more people using the filters. Currently people don’t even know they exist so they don’t even try them out. Now they will discover them by exploring their settings.
2 Likes
@rfindley I observe something strange with the Settings module.
- I have a dialog. When I click
saveboth theon_saveandon_closehandlers are being called. - I have put a
console.trace()in myon_closehandler. It is called from the Settings moduleclose()function which is responsible to close and destroy the dialog.
Is this the intended behavior?
Thank you. I must be blind. I didn’t see there is a separate callback for on_cancel.
1 Like
I’m trying to improve the loading speed of a script that uses WKOF. My goal is for the script to render its content with the first paint of the page using cached data until its ajax completes. However, even if all I do is load the settings stored in WKOF, that already delays things past the first paint. I was wondering if the issue could be solved in the following manner.
Problem: wkof.ready('document') only fires on load which is generally after first paint. The Settings module waits for this, so there is no way to get around this wait if you use WKOF’s settings. However, I need to be able to read the settings to determine how to display things.
Possible Solution: I’m guessing the reason WKOF waits for this event is because the Settings module has gui functionality that can’t be loaded before this. A solution might be to break out only the functionality for programmatically reading/writing settings into a new module SettingsCore without the gui stuff and then have the original Settings module depend on the new module. This should allow the SettingsCore module to avoid waiting on anything other than a single IndexedDB query (the one that loads the stored settings).
Apiv2 also waits on wkof.ready('document') and has additional latency from always doing an ajax request for the /user API endpoint before saying it’s ready, even if the user script does not need to access wkof.user or wkof.Apiv2.user. It would be nice if there were similarly an Apiv2Core that didn’t require waiting for these things.
Reducing this latency would probably also benefit a bunch of other scripts that use WKOF.
P.S. Is there a reason WKOF waits for the load event instead of DOMContentLoaded?
I was also curious about the possibility of a module that would make it easier kind of pattern where you display cached data immediately, then update later with the fresh ajax data.
Something like the following: Expose a function with_remote(modules, update_fn). The second argument is an asynchronous callback function update_fn which is provided by the userscript to update its gui by making API calls. When a userscript calls with_remote('Apiv2', update_fn), WKOF would immediately call update_fn(wkproxy) with an argument wkproxy which implements a subset of the API provided by the global wkof object but which routes all remote calls to the cache, regardless of how old the cached items are. After this call completes[1] and Apiv2 loads, WKOF then calls update_fn(wkproxy) with an object similar to the one before except that api calls actually wait for fresh data, and the results get stored in the cache.
Here is an example of how a userscript might use this functionality:
wkof.with_remote('Apiv2', async function(wkproxy) {
let result = await wkproxy.get_endpoint('summary');
// display data from result in gui
});
The above would display the data from result immediately on page load using the cache and update it once the ajax completes. An advantage of this approach is that most userscripts could be updated to take advantage of this functionality very easily.
[1] To reduce latency, it might be desirable to not wait for the callback to complete with the cache before calling it with the global wkof object. However, this would require userscripts to be more carefully written to avoid a race.
Generally, the 'ready' flag indicates that everything in the module is ready to use. For the Settings module, this includes the ability to open dialogs that may be dependent on the page’s CSS being loaded.
If needed, I could add additional variable states at earlier stages of load, such as:
wkof.set_state('wkof.Settings', 'loaded');
…as soon as the module is loaded. That would allow you to use the load() function for loading settings.
Regarding cached Api data…
The Apiv2 module has a progress_callback option that you can pass to receive data as it is loaded progressively. (And the ItemData module forwards its options argument to the Apiv2 module).
But there are a few caveats:
- I apparently didn’t add an initial call once the cached data is loaded from indexeddb. That’s an easy fix.
- If I did add that initial call, one of the arguments passed to the callback is the total number of records to be loaded. That information is intended for the Progress dialog and doesn’t include the cached data, so it won’t be relevant to the first call.
- The data isn’t cross-linked between endpoints (e.g.
subjects<==>assignments) until all data is loaded.
Adding an additional state that is triggered at the earliest point settings are readable would definitely be helpful. A state trigger when populate_user_cache first starts executing in the initialization of Apiv2 would also be helpful. However, ItemData waits on Apiv2, so in order to use ItemData before any unneeded ajax completes, I guess ItemData could do most of its initialization right after the “populate_user_cache started running” state is triggered and then ItemData could have it’s own state trigger when it finishes that stage of initialization.
For the Api data, maybe I’m misunderstanding, but it looks like progress_callback is only called with information about the number of items that have been retrieved, not their actual data. But let’s suppose it were extended to provide the cache data and then fresh data when it arrives or we add some new option callback which does this. My main use case for this is for modifying existing scripts to make them render the cached data instantly. So, imagine you have a existing typical userscript like:
wkof.include('Apiv2');
wkof.ready('Apiv2').then(fetch_and_update);
async function fetch_and_update() {
let summary = await wkof.get_endpoint('summary');
// display some information based on summary
if (/*summary shows there are pending reviews or lessons*/) {
let assignments = await wkof.get_endpoint('assignments');
// display some information based on both summary and assignments
}
}
With the suggested method I mentioned in my earlier post, converting this script to load instantly with cached data while waiting for ajax is just a two line change:
wkof.include('Apiv2');
wkof.with_remote('Apiv2', fetch_and_update);
async function fetch_and_update(wkof) {
// everything in here is the same
}
Importantly, I don’t need to modify or even really look at the body of the fetch_and_update function. On the other hand, if I were trying to make this change by using some sort of callback option passed to Apiv2, the simplest code I could come up with is below.
wkof.include('Apiv2');
wkof.ready('Apiv2').then(fetch_and_update);
async function fetch_and_update() {
wkof.get_endpoint('summary', {callback: function(summary) {
// display some information based on summary
if (/*summary shows there are pending reviews or lessons*/) {
await wkof.get_endpoint('assignments',{callback: function(assignments) {
// display some information based on both summary and assignments
}});
}
}});
}
So it requires restructuring the body of the fetch_and_update() function into a chain of nested callbacks. In particular, I have to restructure things at each point where there is a call asking wkof for data. If the userscript were instead doing something involving simultaneous API calls using Promise.all(), I think restructuring it to work with callbacks could be even worse.
@rfindley I want to be able to load lzma compressed files. These are binary files. Currently wkof.load_file() supports only strings.
I have made some tests and very little change is need to add binary support.
You need to add a binary parameter to the load_file() signature.
function load_file(url, use_cache, binary) {
Then you need to add a line to the fetch_url() function to request a bytearray. See the comment.
// Retrieve file from server
function fetch_url(){
var request = new XMLHttpRequest();
request.onreadystatechange = process_result;
request.open('GET', url, true);
if (binary) request.responseType = "arraybuffer"; // support binary transfer
request.send();
return fetch_promise;
}
Then you need to update the comment in the published interface.
load_file: load_file, // load_file(url, use_cache, binary) => Promise
And this is it. Support for binary file transfer is added.
The caveat is this is not supported by older browsers. See Mozilla.org.
I have verified that binary files are stored in and retrieved from the cache just fine. I have also verified that adding this parameter does harm calls to wkof.load_file() that don’t supply it.
There is no rush. I can download binary files with my own copy of the load_file() function. I don’t need to wait for you to update anything. It is just that I think this would be a nice addition to the framework.
Instead of 'binary' for the third parameter, I would recommend an 'options'object. Maybe something like:
options: {
response_type: 'arraybuffer'
}
And:
if (options.response_type) request.responseType = options.response_type;
That would support other and future file types. Probably the next most useful would be ‘json’.
In general, ‘options’ objects allow you to continue expanding functionality without worrying about future order and presence of arguments.
1 Like
This looks like a good idea. It shows you have more experience than me with this sort of things.
I have tested your proposed modification. It works as intended. You may merge it whenever you want.
I suggest you use responseType instead response_type for the option name to be consistent with the request attribute name.
Just letting you know. I am canceling my plan to download a binary file. I still think this is a valuable addition to the framework but I know you are busy. I hate to make you work on something that won’t be used. Sorry about this.