Wanikani Open Framework [developer thread]

Thank you! This is all still a bit over my level of understanding, so I’ll just use that snippet. That’s good though, gives me an opportunity to learn more JS.

1 Like

The code below is equivalent, if that helps.

// Load the ItemData module, and run get_items() when it's done loading.
wkof.include('ItemData');
wkof.ready('ItemData').then(get_items);

// Fetch the items from WK (and/or cache)
function get_items() {
    wkof.ItemData.get_items('subjects,assignments')
    .then(process_items);
}

// Separate the items array into srs levels
function process_items(items) {
    // Items is an array of all WK items:
    // [item1, item2, item3, ....]

    // Index the items array by srs stage name, and store in an object
    var by_srs = wkof.ItemData.get_index(items,'srs_stage_name')
    // result:
    // {
    //   "Apprentice 1": [item1, item2, ...],
    //   "Apprentice 2": [item1, item2, ...],
    //   ...
    //   "Burned": [item1, item2, ...]
    // }

    // Get the list of srs categories from the indexed object.
    // Note, if the user doesn't have any items in a particular srs level,
    // that srs name won't be present in the object.
    // Result:  ['Apprentice 1', 'Apprentice 2', ..., 'Burned']
    var srs_names = Object.keys(by_srs);

    // Loop through each srs category, and log the length to the console.
    for (var idx in srs_names) {
        var srs_name = srs_names[idx];
        console.log(srs_name+' => '+by_srs[srs_name].length+' items');
    }
}
1 Like

Aaah, that makes a lot of sense. Thank you for taking the time to type all that out. The way you do things is so efficient, my version would probably be ten times longer and have lots of bugs :sweat_smile:

1 Like

I really need to start playing around with this sometime. Where are you free time?:mag_right:

@rfindley I just installed the released framework instead of the beta, and I’m seeing this:

image

Progress never moves and whenever I refresh it comes back. I assume something is out of sync from using the beta version. Is there anything I need to clear in indexdb or localstorage to get it to load the data from scratch? (I prefer not to fully clear localstorage since other scripts I use have data in them as well.)

If it’s an issue with the beta code, you can try the following, then refresh:

wkof.file_cache.clear();

If the problem continues, post the relevant wkof code and I’ll troubleshoot. (And any errors on the JS console)

Fwiw, I can run the following code on the console, and I’m not seeing an issue:

var summary; wkof.Apiv2.get_endpoint('summary').then(data => summary = data);

Seems to be an issue with the current framework I guess (or I’m doing something wrong).

Here’s what I did:

  1. Opened the WaniKani home page (and saw the stuck popup).
  2. Ran wkof.file_cache.clear(); in the console.
  3. Refreshed and saw the popup with the progress bar moving. After it finished, the popup closed and my script worked as expected (continued to work as expected since it was previously working as well).
  4. Refreshed and saw the popup get stuck again.

There are no errors in the console. As mentioned in step 3, my script works, it’s just the popup that’s acting strangely.

Here’s the relevant code. It’s mostly the code you sent me a few weeks ago and boiler plate UI stuff.

// ==UserScript==
// @name          WaniKani Lesson Hover Details
// @namespace     https://www.wanikani.com
// @description   Show lesson breakdown by type on hover
// @author        seanblue
// @version       1.0.0
// @include       *://www.wanikani.com/*
// @grant         none
// ==/UserScript==

(function() {
    'use strict';

	var style =
		'<style>' +
			'.table { display: table; margin: 0; padding: 0; }' +
			'.row { display: table-row; margin: 0; padding: 0; }' +
			'.cell { display: table-cell; margin: 0; }' +
			'.cell-title { font-weight: bold; padding: 0 10px 0 0; text-align: right; }' +
			'.cell-value { text-align: left; }' +
		'</style>';

	$('head').append(style);

    wkof.include('Apiv2, ItemData');
	wkof.ready('Apiv2, ItemData').then(fetchData);

	function fetchData() {
		var promises = [];
		promises.push(wkof.Apiv2.get_endpoint('summary'));
		promises.push(wkof.Apiv2.get_endpoint('subjects'));

		Promise.all(promises).then(processData);
	}

	function processData(results) {
		var lessonCounts = getLessonCount(results);
		setupPopover(lessonCounts);
	}

	function getLessonCount(results) {
		// Grab the fetched results (same order as the promises array above).
		var summary = results[0];
		var subjects = results[1];

		var lessonCounts = {
			radical: 0,
			kanji: 0,
			vocabulary: 0
		};

		// Pull the list of subject_ids from the lesson list in 'summary'.
		var lessonSubjectIds = summary.lessons[0].subject_ids;
		lessonSubjectIds.forEach(function(subjectId) {
			var item = subjects[subjectId];
			lessonCounts[item.object]++;
		});

		return lessonCounts;
	}

	function setupPopover(lessonCounts) {
		var lessonMenuItem = $('.navbar .lessons a');
		if (lessonMenuItem.length === 0)
			return;

		lessonMenuItem.attr('data-content', getPopoverHtml(lessonCounts)).popover({
			html: true,
			animation: false,
			placement: 'bottom',
			trigger: 'hover',
			template: '<div class="popover review-time"><div class="arrow"></div><div class="popover-inner"><div class="popover-content"><p></p></div></div></div>'
		});
	}

	function getPopoverHtml(lessonCounts) {
		return `<div class="table">
	<div class="row">
    	<div class="cell cell-title">Radicals</div>
    	<div class="cell cell-value">${lessonCounts.radical}</div>
	</div>
	<div class="row">
		<div class="cell cell-title">Kanji</div>
    	<div class="cell cell-value">${lessonCounts.kanji}</div>
	</div>
	<div class="row">
		<div class="cell cell-title">Vocab</div>
    	<div class="cell cell-value">${lessonCounts.vocabulary}</div>
	</div>
</div>`;
	}
})();

Okay, I can replicate the issue with just the following on the JS console:

wkof.Apiv2.get_endpoint('summary'); wkof.Apiv2.get_endpoint('subjects');

I’ll check it out.

Thanks.

That aside, I remember you had a suggestion for a better approach to getting the data I need. My script needs the count of the number of lessons for each subject type. Is this summary + subjects approach the best approach, or is there a different endpoint I should check out? No need to write out the actual implementation, just looking for the most efficient endpoint to use given these requirements.

:point_right: [v1.0.3] - Close progress dialog when no updates from WK server.

2 Likes

What you have now is the best approach for what you’re trying to do.
When I first mentioned a more efficient way, coordination of requests was done in the ItemData layer, which sits on top of Apiv2. Since then, I’ve moved the coordination down into Apiv2, so it’s efficient either way.

Also, ‘summary’ isn’t available via ItemData, since ItemData is solely for actual item info (subjects, assignments, review_statistics, study_materials).

1 Like

@rfindley You have this post which you want people to get redirected to if they don’t have the framework installed. But do you have some pre-canned code you want included in scripts using the framework to do that redirection?

So far, I’ve put the following at the top of Burn Manager:

    if (!window.wkof) {
        alert('Burn Manager script requires Wanikani Open Framework.\nYou will now be forwarded to installation instructions.');
        window.location.href = 'https://community.wanikani.com/t/instructions-installing-wanikani-open-framework/28549';
        return;
    }

I’ve had it pop up with false alarms a couple time, but probably because of my dev environment. I’d like to deploy that snippet slowly before I roll it out into one of my high-volume scripts.

[edit: if the false alarms aren’t due to my dev env, I’ve got bigger problems, because the framework won’t work reliably as-is. So testing this now will be a good thing!]

You’re welcome to continue using that to help me test it out.

I wonder if there are some timing issues at play. For example, if I move the Open Framework below my script using it, my script still works. Perhaps that means the opposite can occur too.

By design, that shouldn’t be an issue… but I wouldn’t rule it out.

Basically, if TamperMonkey is working correctly, all of the synchronous portions of wkof should run first as long as it is first in the run order. And wkof is designed to cache any requests until all of its asynchronous parts are done loading.

I’ve also noticed that scripts sometimes work when wkof runs after. I haven’t investigated it, but I would guess it’s because the client script calls wkof after at least one time increment in the Javascript scheduler.

Having issue with settings. Dialog works fine, but settings don’t seem to save.
according to Dev Console => wkof.settings = {}

Code
    function open_settings() {
        settings_dialog.open();
    }
    function install_settings() {
        settings_dialog = new wkof.Settings({
            script_id: 'srsGrid',
            title: 'SRS Grid V2',
            on_save: process_settings,
            settings: {
                'pg_display': {type:'page',label:'Display',content:{
                    'grp_detail': {type:'group',label:'Review Details',content:{
                        'gridMode': {type:'dropdown',label:'Mode',content:{'left/right':'Left/Right','center/center':'Center/Center'}}
                    }}
                }}
            }
        });
    }
    function process_settings(){
        console.log('Settings saved!');
    }

‘on_save’ only means the user clicked the Save button, and the settings were written to wkof.settings, but not to any kind of storage. It’s currently up to the individual app to write their settings to storage, presumably in their on_save callback:

function process_settings() {
    localStorage.setItem('myscript_settings', JSON.stringify(wkof.settings.myscript));
    console.log('Settings saved!');
}

The reason for not just automatically saving to storage was that I didn’t want to make assumptions about how, where, and when individual scripts would load and save their settings. Maybe scripts would need to load some of their settings early to be able to pre-render stuff? I suppose they could store such critical settings separately

I’m open to adding auto-save.

Gotcha. I was assuming wkof would auto-save everything as json into wkof.settings.srsGrid or something similar.

I could add a flag to the config, like autosave: true. Now to decide whether that should default to true or false… What do you think?

I guess my thought for now is that most users will want to autosave, so I’ll probably default to true. I’ll be out for a few hours, but will push a change one way or the other this afternoon (Eastern time).

@DaisukeJigen,

I’ve added several ways to load and save settings.
Suppose your script is called ‘my_script’, and your settings are stored in `wkof.settings.my_script.
From the options below, most scripts will probably want to use manual load at startup, and automatic save when closing the settings dialog. Additionally, if you want to make programmatic changes to your settings, you’d probably want to use manual save.

Manually save settings

To manually save the contents of wkof.settings.my_script:

    wkof.Settings.save('my_script');

The settings will be saved to file_cache as “wkof.settings.my_script”, which you can confirm via wkof.file_cache.dir.

(alternative)

or, if your settings dialog is already created, you can optionally call save() from the dialog object itself:

    var settings_dialog = new Settings(...);
    ...
    settings_dialog.save();

Manually load settings

To manually load your saved settings into wkof.settings[script_id]:

    wkof.Settings.load('my_script');
(alternative)

or, if your settings dialog is already created, you can optionally call load() from the dialog object itself:

    var settings_dialog = new Settings(...);
    ...
    settings_dialog.load();

Automatically save settings (when Save button pressed)

To automatically save your settings when clicking the ‘Save’ button in your Settings dialog box, you don’t have to do anything. By default, it will now save.

Prevent auto-saving settings (when Save button pressed)

To prevent autosave from occurring, add an autosave: false to your Settings dialog’s configuration.

	settings_dialog = new wkof.Settings({
		script_id: 'my_script',
		title: 'My Script',
		on_save: process_settings,
		autosave: false,  // <------- Prevent autosave
		settings: {
			...
		}
	});

No auto-loading

Since scripts will almost certainly want to load their settings before they ever open a settings dialog, I didn’t add an auto-load to the dialog box. Instead, somewhere at the beginning of your script you should call:

    wkof.Settings.load('my_script').then(process_settings);

The stored settings will be passed as the first argument to process_settings(), or you can access it directly via wkof.settings.my_script.


Does this sound okay?
The changes are uploaded to GreasyFork, so it’s ready to test.

3 Likes