Wanikani Open Framework [developer thread]

Seems to be an issue with the current framework I guess (or I’m doing something wrong).

Here’s what I did:

  1. Opened the WaniKani home page (and saw the stuck popup).
  2. Ran wkof.file_cache.clear(); in the console.
  3. Refreshed and saw the popup with the progress bar moving. After it finished, the popup closed and my script worked as expected (continued to work as expected since it was previously working as well).
  4. Refreshed and saw the popup get stuck again.

There are no errors in the console. As mentioned in step 3, my script works, it’s just the popup that’s acting strangely.

Here’s the relevant code. It’s mostly the code you sent me a few weeks ago and boiler plate UI stuff.

// ==UserScript==
// @name          WaniKani Lesson Hover Details
// @namespace     https://www.wanikani.com
// @description   Show lesson breakdown by type on hover
// @author        seanblue
// @version       1.0.0
// @include       *://www.wanikani.com/*
// @grant         none
// ==/UserScript==

(function() {
    'use strict';

	var style =
		'<style>' +
			'.table { display: table; margin: 0; padding: 0; }' +
			'.row { display: table-row; margin: 0; padding: 0; }' +
			'.cell { display: table-cell; margin: 0; }' +
			'.cell-title { font-weight: bold; padding: 0 10px 0 0; text-align: right; }' +
			'.cell-value { text-align: left; }' +
		'</style>';

	$('head').append(style);

    wkof.include('Apiv2, ItemData');
	wkof.ready('Apiv2, ItemData').then(fetchData);

	function fetchData() {
		var promises = [];
		promises.push(wkof.Apiv2.get_endpoint('summary'));
		promises.push(wkof.Apiv2.get_endpoint('subjects'));

		Promise.all(promises).then(processData);
	}

	function processData(results) {
		var lessonCounts = getLessonCount(results);
		setupPopover(lessonCounts);
	}

	function getLessonCount(results) {
		// Grab the fetched results (same order as the promises array above).
		var summary = results[0];
		var subjects = results[1];

		var lessonCounts = {
			radical: 0,
			kanji: 0,
			vocabulary: 0
		};

		// Pull the list of subject_ids from the lesson list in 'summary'.
		var lessonSubjectIds = summary.lessons[0].subject_ids;
		lessonSubjectIds.forEach(function(subjectId) {
			var item = subjects[subjectId];
			lessonCounts[item.object]++;
		});

		return lessonCounts;
	}

	function setupPopover(lessonCounts) {
		var lessonMenuItem = $('.navbar .lessons a');
		if (lessonMenuItem.length === 0)
			return;

		lessonMenuItem.attr('data-content', getPopoverHtml(lessonCounts)).popover({
			html: true,
			animation: false,
			placement: 'bottom',
			trigger: 'hover',
			template: '<div class="popover review-time"><div class="arrow"></div><div class="popover-inner"><div class="popover-content"><p></p></div></div></div>'
		});
	}

	function getPopoverHtml(lessonCounts) {
		return `<div class="table">
	<div class="row">
    	<div class="cell cell-title">Radicals</div>
    	<div class="cell cell-value">${lessonCounts.radical}</div>
	</div>
	<div class="row">
		<div class="cell cell-title">Kanji</div>
    	<div class="cell cell-value">${lessonCounts.kanji}</div>
	</div>
	<div class="row">
		<div class="cell cell-title">Vocab</div>
    	<div class="cell cell-value">${lessonCounts.vocabulary}</div>
	</div>
</div>`;
	}
})();

Okay, I can replicate the issue with just the following on the JS console:

wkof.Apiv2.get_endpoint('summary'); wkof.Apiv2.get_endpoint('subjects');

I’ll check it out.

Thanks.

That aside, I remember you had a suggestion for a better approach to getting the data I need. My script needs the count of the number of lessons for each subject type. Is this summary + subjects approach the best approach, or is there a different endpoint I should check out? No need to write out the actual implementation, just looking for the most efficient endpoint to use given these requirements.

:point_right: [v1.0.3] - Close progress dialog when no updates from WK server.

2 Likes

What you have now is the best approach for what you’re trying to do.
When I first mentioned a more efficient way, coordination of requests was done in the ItemData layer, which sits on top of Apiv2. Since then, I’ve moved the coordination down into Apiv2, so it’s efficient either way.

Also, ‘summary’ isn’t available via ItemData, since ItemData is solely for actual item info (subjects, assignments, review_statistics, study_materials).

1 Like

@rfindley You have this post which you want people to get redirected to if they don’t have the framework installed. But do you have some pre-canned code you want included in scripts using the framework to do that redirection?

So far, I’ve put the following at the top of Burn Manager:

    if (!window.wkof) {
        alert('Burn Manager script requires Wanikani Open Framework.\nYou will now be forwarded to installation instructions.');
        window.location.href = 'https://community.wanikani.com/t/instructions-installing-wanikani-open-framework/28549';
        return;
    }

I’ve had it pop up with false alarms a couple time, but probably because of my dev environment. I’d like to deploy that snippet slowly before I roll it out into one of my high-volume scripts.

[edit: if the false alarms aren’t due to my dev env, I’ve got bigger problems, because the framework won’t work reliably as-is. So testing this now will be a good thing!]

You’re welcome to continue using that to help me test it out.

I wonder if there are some timing issues at play. For example, if I move the Open Framework below my script using it, my script still works. Perhaps that means the opposite can occur too.

By design, that shouldn’t be an issue… but I wouldn’t rule it out.

Basically, if TamperMonkey is working correctly, all of the synchronous portions of wkof should run first as long as it is first in the run order. And wkof is designed to cache any requests until all of its asynchronous parts are done loading.

I’ve also noticed that scripts sometimes work when wkof runs after. I haven’t investigated it, but I would guess it’s because the client script calls wkof after at least one time increment in the Javascript scheduler.

Having issue with settings. Dialog works fine, but settings don’t seem to save.
according to Dev Console => wkof.settings = {}

Code
    function open_settings() {
        settings_dialog.open();
    }
    function install_settings() {
        settings_dialog = new wkof.Settings({
            script_id: 'srsGrid',
            title: 'SRS Grid V2',
            on_save: process_settings,
            settings: {
                'pg_display': {type:'page',label:'Display',content:{
                    'grp_detail': {type:'group',label:'Review Details',content:{
                        'gridMode': {type:'dropdown',label:'Mode',content:{'left/right':'Left/Right','center/center':'Center/Center'}}
                    }}
                }}
            }
        });
    }
    function process_settings(){
        console.log('Settings saved!');
    }

‘on_save’ only means the user clicked the Save button, and the settings were written to wkof.settings, but not to any kind of storage. It’s currently up to the individual app to write their settings to storage, presumably in their on_save callback:

function process_settings() {
    localStorage.setItem('myscript_settings', JSON.stringify(wkof.settings.myscript));
    console.log('Settings saved!');
}

The reason for not just automatically saving to storage was that I didn’t want to make assumptions about how, where, and when individual scripts would load and save their settings. Maybe scripts would need to load some of their settings early to be able to pre-render stuff? I suppose they could store such critical settings separately

I’m open to adding auto-save.

Gotcha. I was assuming wkof would auto-save everything as json into wkof.settings.srsGrid or something similar.

I could add a flag to the config, like autosave: true. Now to decide whether that should default to true or false… What do you think?

I guess my thought for now is that most users will want to autosave, so I’ll probably default to true. I’ll be out for a few hours, but will push a change one way or the other this afternoon (Eastern time).

@DaisukeJigen,

I’ve added several ways to load and save settings.
Suppose your script is called ‘my_script’, and your settings are stored in `wkof.settings.my_script.
From the options below, most scripts will probably want to use manual load at startup, and automatic save when closing the settings dialog. Additionally, if you want to make programmatic changes to your settings, you’d probably want to use manual save.

Manually save settings

To manually save the contents of wkof.settings.my_script:

    wkof.Settings.save('my_script');

The settings will be saved to file_cache as “wkof.settings.my_script”, which you can confirm via wkof.file_cache.dir.

(alternative)

or, if your settings dialog is already created, you can optionally call save() from the dialog object itself:

    var settings_dialog = new Settings(...);
    ...
    settings_dialog.save();

Manually load settings

To manually load your saved settings into wkof.settings[script_id]:

    wkof.Settings.load('my_script');
(alternative)

or, if your settings dialog is already created, you can optionally call load() from the dialog object itself:

    var settings_dialog = new Settings(...);
    ...
    settings_dialog.load();

Automatically save settings (when Save button pressed)

To automatically save your settings when clicking the ‘Save’ button in your Settings dialog box, you don’t have to do anything. By default, it will now save.

Prevent auto-saving settings (when Save button pressed)

To prevent autosave from occurring, add an autosave: false to your Settings dialog’s configuration.

	settings_dialog = new wkof.Settings({
		script_id: 'my_script',
		title: 'My Script',
		on_save: process_settings,
		autosave: false,  // <------- Prevent autosave
		settings: {
			...
		}
	});

No auto-loading

Since scripts will almost certainly want to load their settings before they ever open a settings dialog, I didn’t add an auto-load to the dialog box. Instead, somewhere at the beginning of your script you should call:

    wkof.Settings.load('my_script').then(process_settings);

The stored settings will be passed as the first argument to process_settings(), or you can access it directly via wkof.settings.my_script.


Does this sound okay?
The changes are uploaded to GreasyFork, so it’s ready to test.

3 Likes

Sounds like that’ll be more than plenty. Thanks.
Probably won’t get a chance to play around until Friday though.

@rfindley The framework shaping up nicely.
I already started porting my script to it and it removed quite a bit of boilerplate code.

There are a few things that would be nice to have:

Scoping for settings keys

Currently the settings generated by defining the menu seem to be only a flat one dimensional hash.
It would be nice if settings could be scoped, either by page, tab or group.
So if I had something like this:

let settings_dialog = new wkof.Settings({
  script_id: "wklc"
  title: "WaniKani Lesson Cap"
  on_save: save_setting
  settings: {
    general: {
      type: "page",
      label: "General",
      content: {
        max_score: { type: "number", label: "Max score", default: 100 }
      }
    },
    scores: {
      type: "page",
      label: "Scores",
      content: {
        apprentice1: { type: "number", label: titleize("apprentice1"), default: 6.0 },
        apprentice2: { type: "number", label: titleize("apprentice1"), default: 3.0 }
      }
    }
  }
});

I would like to get a structure like this:

"wklc": {
  "general": {
    "max_score": 100
  },
  "scores": {
    "apprentice1": 6.0,
    "apprentice2": 3.0
  }
}

This would make it easier to iterate over certain collections of settings for example.

Multi select options

Not sure if this is already in the framework, didn’t look through all of the code yet, but having multiselect as an option would be nice for some settings. The other alternative is to create a lot of checkboxes, which is a bit tedious.


I also noticed that input fields seem to be always of type="text", despite the type being specified as "number" in the settings setup.

Well, that turned out to be a longer post than expected :sweat_smile:

Great feedback… thanks!

I’ve already added some of the changes/additions you mentioned, though I haven’t uploaded them yet since I seem to be making several changes per day.

I’ve added a ‘list’ type (though I may rename that… maybe ‘select’?), which is a non-dropdown <select>. It has support for ‘multi_select:true’, but I still need to test it.

I’ve also added ‘section’, which creates a section header in the dialog (I’ll post pics when I’m back at my PC), and ‘divider’, which is <hr> (Should I just call that type ‘hr’?)

I added an ‘html’ type, which allows you to do advanced or custom stuff beyond what the basic settings can do. It has a ‘pre_open’ callback that you can use to attach event handlers.

Like you, I’ve also thought about structured settings. I definitely don’t want to tie the settings data structure to the dialog structure, because I have several dialogs that I know won’t work well that way. I’ve thought about using a dotted field name (e.g. ‘general.max_score’). Currently, you can actually do that and it will do something like:

setting.myscript[‘general.max_score’]

Then you could iterate via:

var general_settings = Object.keys(settings.myscript).filter(name => name.match(/^general\./) !== null);

But that’s messy. So, I’m trying to think of a good way to parse a nested field name without the code getting overly complicated. I have some ideas, and will probably work on that tonight.

Oh, and ‘number’ being ‘text’ was a total oversight. I’ll fix that tonight. And maybe I’ll add type:'input' that accepts a subtype (e.g. subtype:'password'), so I can support all the other less common types without bloating the code.

1 Like

I’m still working on this, and will hopefully be done in the next day or two. Adding the scoping really messed up the code, but I’ve got it mostly straightened up again, and I think it’s going to be a lot more powerful now. I added a “path” parameter that overrides the default flat scoping.

Using your example, path can be something simple like “@scores.apprentice1”, where ‘@’ is shorthand for wkof.settings.your_script.

But you can also use nearly any expression in the path, which opens up some interesting possibilities.
In the example below, I have a dropdown that lets you choose between some presets, and two checkboxes that modify the values inside the active preset. (i.e. presets[active_preset].some_setting)

wkof.settings.my_script = {
    active_preset: 0,
    presets: [
        {reading2meaning: true, meaning2reading: false},
        {reading2meaning: false, meaning2reading: true},
        {reading2meaning: true, meaning2reading: true}
    ]
}

var config = {
    active_preset: {type: 'dropdown', refresh_on_change: true, content: {
        '0': 'JP to EN',
        '1': 'EN to JP'
        '2': 'Both'
    },
    reading2meaning: {
        type:'checkbox',
        path:'@presets[@active_preset].reading2meaning'
    },
    meaning2reading: {
        type:'checkbox',
        path:'@presets[@active_preset].meaning2reading'
    }
}

The ‘refresh_on_change: true’ tells it to update all of the controls when the preset is changed. All automatic!

2 Likes

I really appreciate the work you’re putting into this

2 Likes

@valeth, @Kumirei, @seanblue, @acm2010, @DaisukeJigen,

I’m ready to push an update for the wkof.Settings module, and want to make sure I don’t break anyone’s script. To the best of my knowledge, I think the only breaking change is:

  • The parameters passed to any “on_change” functions.
    • before: on_change(value, config)
    • after: on_change(name, value, config)
      • name - The name of the particular setting’s node in the configuration object.
      • value - The new value of the setting.
      • config - The contents of the of the particular setting’s node in the configuration object.

I could have just put the name parameter at the end to avoid breaking the interface, but having it first makes it consistent with the rest of the interface, and as far as I’m aware, no one has a released script that uses the on_change callbacks yet.

…which brings me to my next point. It would be great to have a registry of scripts that use the framework, so I can look at people’s code before making changes to make sure I don’t break anything. And, in some cases, maybe I can do some testing just to make sure.

So, if you’re using the framework, would you all mind posting a link to any of your released scripts? Also, if I have any big updates coming up, I’ll post here beforehand if I think there’s much risk of anything breaking.

Anyway, I haven’t uploaded the changes yet. I need to do some “quality control” checks beforehand. But I’ll be posting it within the next 18 hours or so.


Okay, so some of the additional changes to the Settings module:

Settings dialog background

I noticed that my (existing) Ultimate Timeline tends to grab certain mouse events when trying to resize a Settings dialog. So, I made a semi-transparent (i.e. tinted) full-screen background that will catch any mouse actions that fall through while resizing or moving a dialog. The background essentially makes the settings dialogs modal. The background is optional (ON by default), and can be disabled by adding background:false to your Settings configuration object. The background also uses a reference counter, so if more than one dialog is open, the background will stay open until the counter drops back down to zero.

image

Structured wkof.settings.my_script (i.e. “path” parameter)

Originally, all settings saved by the Settings dialog would be placed directly under your script’s sub-object:

wkof.settings.my_script = {
    setting1: "value1",
    setting2: "value2",
    ...
}

Now, each setting in your dialog configuration can add a path field, which can evaluate complex expressions.
Suppose you want a non-flat structure in your script’s settings:

wkof.settings.my_script = {
    active_preset: '0',
    presets: [
        '0': {name:'Preset #1', setting1:'value1', setting2: 'value2'},
        '1': {name:'Preset #2', setting1:'value1', setting2: 'value2'},
    ]
}

You can now add a path expression to any setting your dialog configuration:

    setting1: {
        path: '@presets[@active_preset].setting1'
        ...
    }
});

The “@” is shorthand for “wkof.settings.my_script.”, where “my_script” is your script name, of course.

First, I run a path.replace() on the “@”, then I do an eval(). So, you can technically use any valid left-hand side of an assignment expression. Other examples might include:

    path:'@group1["setting_"+@index]'

which translates to:

    wkof.settings.my_script.group1["setting_"+wkof.settings.my_script.index]

New types

There’s now support for:

  • A ‘list’ type with a multi:true for selecting mulitple values.
  • A ‘section’ type that draws a section header bar.
  • A ‘divider’ type that is an <hr> tag.
  • An ‘input’ type, with ‘subtype’ support, so you can do things like <input type="password">
  • An ‘html’ type that lets you add arbitary inline html. You can then use the new global “pre_open” callback to attach any event handlers or whatever.

Automatic refresh

If you have a setting that affects other settings, the new “refresh_on_change:true” can be used to trigger the dialog to update based on the current contents of wkof.settings.my_script.

// Somewhere Inside your dialog configuration:
var dialog = new wkof.Settings({
    ...
    active_preset: {type:'dropdown', refresh_on_change:true},
    setting1: {type: 'checkbox', path:'@presets[@active_preset]'},
    ...
}

In this example, when you change the active preset, the “refresh_on_change” will cause "setting1" to be re-evaluated based on the new preset.

In the example, “setting1” changed because its “path” referred to the active_preset. But you could also use an "on_change" event to modify other settings in a more complex way.

1 Like