[Userscript] WaniKani Open Framework Additional Filters (Recent Lessons, Leech Training, Related Items, and more)

[seanblue]

This is a third-party script/app and is not created by the WaniKani team. By using this, you understand that it can stop working at any time or be discontinued indefinitely.

What is it?

This script adds additional filters for the WaniKani Open Framework. This means that other scripts that use the framework will be able to use these filters. I think this will be particularly useful for the Self-Study Quiz script, as these filters will allow you to customize which items you want to study.

Every filter is available by default. You can always choose which filters to use within the Self-Study script, but as more filters are added, the Self-Study script’s settings page could become cluttered. Therefore, in the main WaniKani menu, you can go to Scripts > Settings > Additional Filters to remove the ones you never want to use.

How to Install?

1. As always, make sure you have a script manager installed. If you don’t, follow these instructions.
2. This script depends on the WaniKani Open Framework. Make sure you have that installed by following these instructions.
3. Get the script here: WaniKani Open Framework Additional Filters
   Make sure to place this script immediately after the WaniKani Open Framework in your script manager.
4. (Optional) Install Self-Study Quiz to get the most out of using this script.

Supported Filters

Recent Lessons

Filter items to show lessons taken in the last X hours. You can change the number of hours to use for the filter. This filter will allow you to easily drill items you have just taken lessons for.

Leech Training

Only include leeches. Formula: incorrect / currentStreak^1.5 >= X. This is the same formula used by hitechbunny’s various leech scripts, so using the default leech threshold of X=1 should give you roughly the same leeches. The higher the value, the fewer items will be included as leeches.

Don’t Show Items Coming Up For Review

Only include items that have at least X% of their SRS interval remaining. This could be used in combination with other filters so that you don’t interfere with the SRS too significantly.

Valid values are from 0 to 100. For example:
75: At least 75% of an item's SRS interval must be remaining. This means for Apprentice 1 (4 hours interval), the item would only be included if there is at least 3 hours until the review. But for Apprentice 2 (8 hours interval), the item would only be included if there is at least 6 hours until the review. And so on.

Failed Last Review

Only include items where the most recent review was failed. The input allows you to only include failed items whose last review was in the last X hours.

Related Items

Only include items that contain at least one of the given kanji. You specify the list of kanji you want by just including them sequentially (no spaces needed). You can optionally add a space and a hyphen (-) followed by another list of kanji that you want to exclude.

Examples:

• 金: Returns all items containing the kanji 金.
• 金髪 -曜: Returns all items containing the kanji 金 or 髪, but not 曜.

I recommend using this filter with a kanji filter to directly quiz the kanji you include in this filter, or with a vocab filter to quiz similar items that you often get mixed up.

Additional Requests?

I’m more than happy to add additional filters, so let me know if you have any requests.

Revision History

• 1.3.1 - Only redirect users to install the Open Framework if they want to.
• 1.3.0 - Add Related Items filter.
• 1.2.0 - Add Failed Last Review filter.
• 1.1.0 - Add Time Until Review filter.
• 1.0.1 - Minimize API calls.
  • Requires WaniKani Open Framework version 1.0.18 or higher.
• 1.0.0 - Initial release (includes Recent Lessons and Leech Training filters).

[jprspereira]

THANK YOU FOR THIS!

Seriously, I’m 3 days away from the fast levels and this script will save me a lot of time and headaches!

[Powerpuncher]

Omg I’ll love you forever, thank you so much!

[rfindley]

Fantastic! I’m so glad to see some custom filters being added so soon.

@seanblue, I plan to add the Promise feature tonight.

Re: “Don’t Show Items Coming Up For Review”, you could temporarily hard-code the SRS intervals to achieve your 75%. I think this filter is important enough to get it out there early, and adjust the code later if WK makes your requested change. (And actually, I would just like to see an /srs endpoint that details the exact SRS intervals. How nice would that be!? )

[seanblue]

@rfindley

Re: “Don’t Show Items Coming Up For Review”, you could temporarily hard-code the SRS intervals to achieve your 75%. I think this filter is important enough to get it out there early, and adjust the code later if WK makes your requested change.

I was thinking the same thing. I’m hoping to work on it within the next week or so, unless the WaniKani staff indicates that my request will be fulfilled quickly.

I’m considering allowing people to specify different percentages per SRS level. So they could specify 90% for Enlightened, but only 50% for Guru 1 (for example). The best idea I have for that is to use filter_value_map and some string parsing. Two ideas I have are:

• Name based: { appr1: 0.2, appr2: 0.2, appr3: 0.4, appr4: 0.4, guru1: 0.6, rest: 1.0 }.
• SRS Level Index based: [0.2, 0.2, 0.4, 0.4, 0.6, 1.0] // Last value is used for all higher SRS levels.

Is there anything available in the framework to make this easier and make it so users don’t have to write complex details in the input? Or perhaps I’m just overcomplicating it. I did something similar to this in my hacked-together version that I made for the old Self Study script, but its usefulness was questionable. Perhaps it’s enough to just use this in combination with the existing SRS Level filter. So you can exclude SRS levels entirely and use the single specified percentage for the rest.

[rfindley]

I imagine most users will just want to pick a preset, so you could make a dropdown in the filter, and have a separate interface (available via the WK menu) for people that want to twiddle with specific settings.

filter_value_map is for converting the filter value to a different format, like “1-5” into an array of levels, etc. So yeah, that’s probably appropriate for interpreting the field values. The point is that filter_value_map gets called only once, whereas filter_func gets called for every item, so you definitely want to pre-calculate as much as you can in filter_value_map.

[rfindley]

@seanblue,

FYI, a useful tip if you’re interested:

Obviously, ItemData needs to be loaded before you can register your filters. But suppose you’re on a Wanikani page where no script is using ItemData, and thus filters aren’t being used. Or, like Self-Study Quiz, it doesn’t include() ItemData until you actually open the quiz dialog, which reduces workload as a page loads.

So, the dilemma is, how do you know if another script is using ItemData? And here’s where the tip comes in.

In your case, you don’t have to do an include(). You can just do a ready(). The ready() will only trigger when another script does the include().

[seanblue]

So is the idea that other scripts can’t trigger one of my filters without having already loaded the ItemData module? So I don’t need to load that module myself just to register my filters since they’ll always be registered before they need to be used?

I feel like I just repeated what you wrote, but I want to make sure I understood correctly.

[rfindley]

Yes, that’s correct.

[rfindley]

@rfindley

Yes, that’s correct.

As long as you register them immediately upon receiving the ‘ready’ event. i.e.:

wkof.ready('ItemData').then(register_filters);

function register_filters() {
    wkof.ItemData.registry.sources.wk_items.filters.my_filter = {
        ...
    };
}

[seanblue]

I don’t think I can do that since I’m using the Settings module to determine whether or not to register each filter. If the ItemData module’s promise resolves before the Settings module’s promise, there could be a timing issue and some script may try to apply filters before mine have been registered.

[rfindley]

True. How about something like this:

---

Filter add-on script:

    wkof.ready('ItemData')
    .then(pause_registry_event)
    .then(install_filters)
    .then(unpause_registry_event);

    function pause_registry_event() {
        wkof.ItemData.pause_registry_event(true);
    }

    function install_filters() {
       return promise_of_some_kind();
    }

    function unpause_registry_event() {
        wkof.ItemData.pause_registry_event(false);
    }

---

Filter user (like Self-Study Quiz):

    wkof.on('wkof.ItemData.registry_ready').then(do_something);

[seanblue]

Will that work if the consumer does ready on ItemData instead of ItemData.registry? If so that should probably be fine.

On a related note, maybe using a setting for this doesn’t make sense. Have you considered allowing filters to be grouped together (allowing multiple groups per source)? Then on the Self Study script you could group them together in a subsection or collapsible section so the UI doesn’t get cluttered.

[rfindley]

@seanblue

Will that work if the consumer does ready on ItemData instead of ItemData.registry?

I suppose we could flip this around. Have ItemData send out an event for modules to register stuff. Then, after any registering is done, do the regular ItemData event.

[rfindley]

@seanblue

Have you considered allowing filters to be grouped together

I want the average scripter to be able to build a decent GUI with minimal work, which is why I’m avoiding making the filter registry more complex (or, at least, trying not to)

I’m considering moving the whole “Item Sources” code out of Self-Study, and just make an “item_sources” type in the Settings module. I suspect that’s the only way anyone is ever going to create a complex settings dialog like Self-Study has, because I don’t think the average coder wants to do something that complex.

Basically, they’d only have to add:

var dialog = new wkof.Settings({
    ...
    item_src: {
      type: 'item_sources',
      path: '@path_to_settings',
    }
    ...
});

And the following would be added to their dialog box (plus any additional registered filters, of course):

[seanblue]

@rfindley

I suppose we could flip this around. Have ItemData send out an event for modules to register stuff. Then, after any registering is done, do the regular ItemData event.

I’m not entirely sure what you mean by this. Right now, does wkof.ready('ItemData') resolve before all of its subcomponents have resolved?

[rfindley]

No… There aren’t currently any subcomponents of ItemData (other than Apiv2).

I’m suggesting adding the registration as a preliminary stage.

1. ItemData finishes its first-pass evaluation in the browser
2. It launches a registration stage by setting a wkof.ItemData.registry state to ‘ready’.
3. Any client scripts listening for that event will proceed with registration, optionally asking ItemData to pause on this stage while they do some work.
4. When clients complete any registration, if they asked for a pause, they release the pause.
5. ItemData sees that registration is complete, and signals the normal ready('ItemData')

[rfindley]

I’ve pushed the changes to github and greasyfork. Now you can:

// Listen for ItemData to tell us when to register our filters
wkof.wait_state('wkof.ItemData.registry', 'ready').then(register_filters);

// Register the filters
function register_filters() {
    // Tell ItemData to wait while we load our settings
    wkof.ItemData.pause_ready_event(true);

    // Load our settings
    wkof.Settings.load('my_settings')
    .then(function(){
        // Settings are loaded.  Now register the filters.
        // [...]
        // And finally, tell ItemData it can resume (i.e. send out its final 'ready')
        wkof.ItemData.pause_ready_event(false);
    });
}

// Most scripts will be listening for this event.
// At this point, all filters should be registered.
wkof.ready('ItemData')
.then(function(){
    // Scripts can proceed knowing that the registry is populated.
});

[edit: correction… I hadn’t yet pushed to github, but now I have]

[seanblue]

In case you’re curious, I had to do a little more than you suggested to get it working completely. Since I also want to be able to edit the filter settings in the menu if nothing loads ItemData, I actually had to call the code loading the settings in two different ways. The first way is as you suggested, but the menu version doesn’t wait on wkof.ItemData.registry being ready. Plus then I had to add some logic to not load the settings twice unnecessarily or register the filters more than necessary.

If nothing else, it was good practice with JavaScript promises. I was getting a bit rusty with promises in general, and it was my first time using the ones built into JavaScript instead of the old jQuery versions.

[zdennis]

@seanblue

@seanblue

Don’t Show Items Coming Up For Review
The idea for this filter is to not show items that will be coming up for review soon. This is so you don’t completely mess up the SRS by self studying.

I’d like this to be percentage based, so that it works dynamically regardless of SRS level. For example, if you say 75% of the waiting period must be remaining, then it would only include Apprentice 1 items that have at least 3 hours until their next review (since the interval is 4 hours), Apprentice 2 items that have at least 6 hours until their next review, and so on.

This filter would currently be difficult to write, but I have a request out to WaniKani to add the necessary data to the API so this filter can be added easily.

// ==UserScript==
// @name          WaniKani Open Framework Additional Filters
// @namespace     https://www.wanikani.com
// @description   Additional filters for the WaniKani Open Framework
// @author        seanblue
// @version       1.0.0
// @include       *://www.wanikani.com/*
// @grant         none
// ==/UserScript==

(function() {
	'use strict';

	var settingsDialog;
	var settingsScriptId = 'additionalFilters';
	var settingsTitle = 'Additional Filters';

	var filterNamePrefix = 'additionalFilters_';
	var recentLessonsFilterName = filterNamePrefix + 'recentLessons';
	var leechTrainingFilterName = filterNamePrefix + 'leechTraining';
	var SecureSrsFilterName = filterNamePrefix + 'secureSrs';

	var supportedFilters = [recentLessonsFilterName, leechTrainingFilterName, SecureSrsFilterName];

	var defaultSettings = {};
	defaultSettings[recentLessonsFilterName] = true;
	defaultSettings[leechTrainingFilterName] = true;
	defaultSettings[SecureSrsFilterName] = true;

	var recentLessonsHoverTip = 'Filter items to show lessons taken in the last X hours.';
	var leechesSummaryHoverTip = 'Only include leeches. Formula: incorrect / currentStreak^1.5.';
	var leechesHoverTip = leechesSummaryHoverTip + '\n * The higher the value, the fewer items will be included as leeches.\n * Setting the value to 1 will include items that have just been answered incorrectly for the first time.\n * Setting the value to 1.01 will exclude items that have just been answered incorrectly for the first time.';
	var secureSrsHoverTip = "Exclude Items which time to next review is lower then the percentage of the SRS Level Time"; // TODO: Explain this so a human can understand it.

	var msToHoursDivisor = 3600000;

    var WKTimesInMinutes = {1:240,
                        2:480,
                        3:1440,
                        4:4320,
                        5:10080,
                        6:20160,
                        7:43800,
                        8:175200};

	if (!window.wkof) {
		alert('WaniKani Open Framework Additional Filters requires WaniKani Open Framework.\nYou will now be forwarded to installation instructions.');
		window.location.href = 'https://community.wanikani.com/t/instructions-installing-wanikani-open-framework/28549';
		return;
	}

	wkof.include('Menu, Settings, ItemData');

	wkof.ready('Menu').then(installMenu);
	wkof.ready('Settings').then(installSettings);

	function installMenu() {
		wkof.Menu.insert_script_link({
			script_id: settingsScriptId,
			submenu: 'Settings',
			title: settingsTitle,
			on_click: openSettings
		});
	}

	function openSettings() {
		settingsDialog.open();
	}

	function installSettings() {
		var settings = {};
		settings[recentLessonsFilterName] = { type: 'checkbox', label: 'Recent Lessons', hover_tip: recentLessonsHoverTip };
		settings[leechTrainingFilterName] = { type: 'checkbox', label: 'Leech Training', hover_tip: leechesSummaryHoverTip };
		settings[SecureSrsFilterName] = { type: 'checkbox', label: 'Secure SRS', hover_tip: secureSrsHoverTip };



		settingsDialog = new wkof.Settings({
			script_id: settingsScriptId,
			title: settingsTitle,
			on_save: saveSettings,
			settings: settings
		});

		settingsDialog.load().then(function() {
			wkof.settings[settingsScriptId] = $.extend(true, {}, defaultSettings, wkof.settings[settingsScriptId]);
			saveSettings();
		});
	}

	function saveSettings(){
		settingsDialog.save().then(function() {
			wkof.ready('ItemData').then(registerFilters);
		});
	}

	function registerFilters() {
		supportedFilters.forEach(function(filterName) {
			delete wkof.ItemData.registry.sources.wk_items.filters[filterName];
		});

		if (wkof.settings[settingsScriptId][recentLessonsFilterName])
			registerRecentLessonsFilter();

		if (wkof.settings[settingsScriptId][leechTrainingFilterName])
			registerLeechTrainingFilter();

		if (wkof.settings[settingsScriptId][SecureSrsFilterName])
			registerSecureSRSTrainingFilter();
	}

	// BEGIN Recent Lessons
	function registerRecentLessonsFilter() {
		wkof.ItemData.registry.sources.wk_items.filters[recentLessonsFilterName] = {
			type: 'number',
			label: 'Recent Lessons (hours)',
			default: 24,
			filter_func: recentLessonsFilter,
			set_options: function(options) { options.assignments = true; },
			hover_tip: recentLessonsHoverTip
		};
	}

	function recentLessonsFilter(filterValue, item) {
		if (item.assignments === undefined)
			return false;

		var startedAt = item.assignments.started_at;
		if (startedAt === null || startedAt === undefined)
			return false;

		var startedAtDate = new Date(startedAt);
		var timeSinceStart = Date.now() - startedAtDate;

		return (timeSinceStart / msToHoursDivisor) < filterValue;
	}
	// END Recent Lessons

	// BEGIN Leeches
	function registerLeechTrainingFilter() {
		wkof.ItemData.registry.sources.wk_items.filters[leechTrainingFilterName] = {
			type: 'number',
			label: 'Leech Training',
			default: 1,
			placeholder: 'Leech Threshold',
			filter_func: leechTrainingFilter,
			set_options: function(options) { options.review_statistics = true; },
			hover_tip: leechesHoverTip
		};
	}

	function leechTrainingFilter(filterValue, item) {
		if (item.review_statistics === undefined)
			return false;


		var reviewStats = item.review_statistics;
		var meaningScore = getLeechScore(reviewStats.meaning_incorrect, reviewStats.meaning_current_streak);
		var readingScore = getLeechScore(reviewStats.reading_incorrect, reviewStats.reading_current_streak);

		return meaningScore >= filterValue || readingScore >= filterValue;
	}

	function getLeechScore(incorrect, currentStreak) {
		return incorrect / Math.pow((currentStreak || 0.5), 1.5);
	}
	// END Leeches

    // BEGIN SecureSRS
	function registerSecureSRSTrainingFilter() {
		wkof.ItemData.registry.sources.wk_items.filters[SecureSrsFilterName] = {
			type: 'number',
			label: 'Secure SRS',
			default: 75,
			placeholder: 'SecureSRS Percentage',
			filter_func: secureSRSFilter,
			set_options: function(options) { options.assignments = true; },
			hover_tip: secureSrsHoverTip
		};
	}

	function secureSRSFilter(percentage, item) {
		if (item.assignments === undefined)
			return false;

		var reviewLevel = item.assignments.srs_stage;
        if (reviewLevel === 9) return true;
		var reviewAvailable_at = item.assignments.available_at;
		return isUnderThreshold(reviewLevel,reviewAvailable_at, percentage);
	}

    function isUnderThreshold(Level, nextReviewDate, percentage) {
        var difference = (new Date(nextReviewDate).getTime() - new Date().getTime()) / 60000;
        var threshold =  WKTimesInMinutes[Level] * percentage / 100;
        var ret = threshold < difference;
		return ret;
	}

	// END SecureSRS
})();

I modified your code to add this functionality. Maybe you could use it or parts of it