tl;dr: WaniKani has lag issues during reviews, in some cases 100+ ms between questions, even without UserScripts. With several scripts, it can easily be double that. Lesson screens have even higher latency. This post explains the causes and how WaniKani’s staff could fix them, improving latency by ~6x for everyone during reviews and ~10x in lessons. A UserScript which implements many of these fixes is below, but it would be better for the fixes to be natively integrated into WaniKani. To that end, a standalone backwards compatible* permissively licensed drop in replacement for jStorage is provided below with extensive QUnit tests.
Below is a chart showing the time it takes between clicking submit until the webpage updates to show the result after each review item.** There are 6 different versions of WaniKani that were tested. In each version, the submit button was automatically clicked 40 times, resulting in some but not all of the 600 items in the review queue being completed. To ensure consistency, the review queue and order of the items reviewed was exactly the same in each test. The base scenarios tested were WaniKani as it is today without any modifications (column 1 in the chart) and WaniKani today with a bunch of common UserScripts (column 4).
Two groups of performance improvements were tested: replacing the jStorage JavaScript library used by WaniKani with an implementation that only stores data asynchronously (columns 2 and 5) and doing that plus several miscellaneous changes discussed in more detail below (columns 3 and 6). As the chart shows, you get an ~3x latency improvement from async jStorage and an additional ~2x latency improvement from also doing the other changes.
Here is some more detail about the changes that would improve things significantly:
- async jStorage: WaniKani itself and UserScripts heavily rely on the jStorage library. However, every time you save something in jStorage, it saves not just what you changed but everything else stored in it too. So if you have 600 outstanding reviews, every single update to any key in jStorage requires the browser to do JSON serialization of ~256kb of data which is slow. This can be fixed by modifying jStorage to only store changes in memory and only flush those changes to sessionStorage asynchronously when the visibility of the window changes. Doing this still ensures the changes get saved before the window is closed/navigated and removes the lag caused by saving. The “async jStorage” modification I tested also removes the deep copy that jStorage does when storing items in its cache, eliminating the need to do JSON serialization and deserialization of the review queue (~256kb of data) each time a review item is completed. The deep copy was not necessary as neither WaniKani nor any common scripts rely on it during reviews.***
-
don’t use the jQuery
:visibleselector or any other style/layout access: Using it forces the browser to immediately recalculate styles and possibly redo layout. Some of this recalculation would have been done later anyway, but some of the work gets thrown out as additional changes are made to the DOM. Instead, keep track of the visibility of the elements that need to be shown/hidden in variables and query the variables instead. -
never use
html(),parseHTML(), or similar functionality There are several uses of the jQueryhtml()function in WaniKani’s code that are just setting the plain text content of various things and could usetext()instead, which is faster. For other uses, it is usually faster and cleaner to construct the DOM tree manually or to use reusable components such as React or another framework. In cases where parsing html is absolutely necessary, such as when rendering mnemonics, the mnemonic should ideally be parsed in the background before it is needed usingrequestIdleCallback(). -
don’t tell jQuery to do something that results in it doing nothing If you tell jQuery to animate something to a state it is already in, there is a significant performance cost even though jQuery ultimately doesn’t change anything on the page. Examples of this on WaniKani include sometimes scrolling to a position the window is already in and fading out the loading screen after every answer, even though the window is already scrolled and the loading screen is already hidden. Also, calling
hide()/show()on an already hidden/shown element has performance problems too because it triggers style recalculation. When fixing these types of problems, one needs to be careful that one does not introduce additional checks which would force the browser to calculate the style or page layout. (For example, it would be bad to fix this by explicitly checking the visibility of the loading screen in the DOM. See point 3.) - don’t artifically delay the loading screen fading out Currently, the fade out of the loading screen is artificially delayed according to a timer instead of immediately disappearing when the page is ready. This suggestion doesn’t affect the times in the charts, just the initial loading time of the page.
- Use the keydown event instead of the keyup event for keyboard navigation in lessons. This shaves 15-75ms off each keyboard navigation in lessons.
- Configure your frameworks to avoid tearing down and recreating page elements When the page needs to change, it is easier to write correct code that completely trashes and regenerates a large swath of the DOM instead of only updating the DOM elements that actually changed. Thankfully, several frameworks, such as React, can automate this process, but sometimes need to be babied a bit in order to convince them that something really doesn’t need to be udpated. Luckily, there are browser extensions for React that make this process pretty straightforward by letting you see what it decided to update / not update and why. Note that this fix was deemed to difficult for me to implement/benchmark and is not included in the charts.
Below are the charts for the latency of moving between lesson screens. Due to lack of time, I only included benchmarks for WaniKani without scripts and didn’t separate out async jStorage only vs. async jStorage + all the other improvements.
For the lesson screen benchmarks above, there was only one item in the lesson queue, which was repeatedly scrolled through without refreshing the page. So even though the queue size is far smaller in these lessons tests, the latency of switching lesson screens is ~2x worse compared to submitting a review in the review tests.****
UserScript that fixes some of these issues
// ==UserScript==
// @name WK Load Faster
// @namespace est_fills_cando
// @version 0.5
// @description Brings the loading screen down more quickly, removing the mandatory 1.8 second delay.
// @author est_fills_cando
// @match https://www.wanikani.com/review/session
// @match https://www.wanikani.com/lesson/start
// @match https://www.wanikani.com/lesson/session
// @grant none
// @run-at document-start
// ==/UserScript==
///
/// Speeding up jQuery, jStorage, the loading screen, and keyboard navigation
///
(function() {
'use strict';
function main() {
window.waitProperty(window, 'jQuery', proxy_jquery);
waitProperty(window, 'jQuery', function (jQuery) {
waitProperty(jQuery, 'jStorage', replace_jstorage);
});
window.waitProperty(window, 'jQuery', function(jQuery) { jQuery(keydownNavigation) });
}
// WK doesn't make the loading screen removal method global in lessons for some reason
// so we patch jQuery with a proxy that lets us modify the behavior of the method in
// both lessons/reviews by detecting its calls to jQuery and modifying the behavior of jQuery
// to do what we want
// We also patch several other jQuery methods to give faster speed.
async function proxy_jquery() {
let pt = window.jQuery.prototype;
// returns true if given jQuery object is the loading screen
function is_loading_screen(jqobj) {
return jqobj.length == 1 && ['loading-screen','loading'].includes(jqobj.get(0).id);
}
// don't artifically delay things relayed to the loading screen
const old_delay = pt.delay;
pt.delay = function() {
if (is_loading_screen(this))
return this;
else
return old_delay.call(this,...arguments);
}
// hide the loading screen immediately when it is no longer needed instead of a
// gradual fade and don't waste time trying to fade it out again if we have already
// hidden it
const old_fadeOut = pt.fadeOut;
let alreadyHidden = false;
pt.fadeOut = function() {
if (is_loading_screen(this)) {
if (!alreadyHidden) {
this.hide();
alreadyHidden = true;
}
return this;
} else {
return old_fadeOut.call(this,...arguments);
}
}
// disable this type of animated scrolling because usualy animated scrolling is triggered
// by other things and this is invoked even when there is no need to scroll wasting time
const old_animate = pt.animate;
pt.animate = function(properties) {
let names = Object.getOwnPropertyNames(properties);
if (names.length === 1 && names[0] == 'scrollTop' && properties[names[0]] == 0)
return this;
else
return old_animate.call(this, ...arguments);
}
// cache visibility of certain elements to avoid making the browser figure it out
const visCache = new Map();
const vItems = Object.freeze(['information', 'last-items', 'item-info']);
for (let item of vItems) {
visCache.set(item, false);
}
function updateVisCache(jqobjs, vis) {
for (let el of jqobjs) {
if (vItems.includes(el.id))
visCache.set(el.id, vis);
}
}
const old_is = pt.is;
pt.is = function(selector) {
if (selector === ':visible' && this.length === 1 && vItems.includes(this.get(0).id)) {
let c = visCache.get(this.get(0).id) && visCache.get('information');
//if (old_is.call(this, ...arguments) !== c) throw 'bad cache';
return c;
} else {
return old_is.call(this, ...arguments);
}
}
const old_hide = pt.hide;
pt.hide = function() {
updateVisCache(this, false);
return old_hide.call(this, ...arguments);
}
const old_show = pt.show;
pt.show = function() {
updateVisCache(this, true);
return old_show.call(this, ...arguments);
}
// setting innerHTML is slightly faster if we check for non-html content
// and set that using textContent
const textOnly = /^[^<>"'&]*$/u;
let old_html = pt.html;
pt.html = function (html) {
const t = typeof html;
if (t === 'number' || t === 'string' && html.match(textOnly)) {
this.each( function () { this.textContent = html; } );
} else {
return old_html.call(this,...arguments);
}
};
}
function keydownNavigation() {
if (window.location.pathname.startsWith('/lesson')) {
function keyDownToUp(options) {
let alreadySent = false;
let start = null;
document.addEventListener('keydown', function(evt) {
if (evt.key == options.key && !alreadySent && evt.target.tagName.toLowerCase() != 'input') {
alreadySent = true;
let fakeEvt = new KeyboardEvent('keyup',options);
fakeEvt.wklkd_fakeevt = true;
document.querySelector('#next-btn').dispatchEvent(fakeEvt);
}
}, true);
document.body.addEventListener('keyup', function(evt) {
if (evt.key == options.key && !evt.wklkd_fakeevt && alreadySent) {
alreadySent = false;
evt.stopPropagation();
evt.preventDefault();
return false;
}
}, true);
}
keyDownToUp({'key':'Enter', code: 'Enter', which:13, location: 0, keyCode:13, bubbles:true});
keyDownToUp({'key':'ArrowLeft', code: 'ArrowLeft', which:37, location: 0, keyCode:37, bubbles:true});
keyDownToUp({'key':'ArrowRight', code: 'ArrowRight', which:39, location: 0, keyCode:39, bubbles:true});
keyDownToUp({'key':'a', code: 'Keya', which:65, location: 0, keyCode:65, bubbles:true});
keyDownToUp({'key':'d', code: 'Keyd', which:68, location: 0, keyCode:68, bubbles:true});
keyDownToUp({'key':'j', code: 'Keyj', which:74, location: 0, keyCode:74, bubbles:true});
keyDownToUp({'key':'q', code: 'Keyq', which:81, location: 0, keyCode:81, bubbles:true});
keyDownToUp({'key':'g', code: 'Keyg', which:71, location: 0, keyCode:71, bubbles:true});
}
}
// helper method for waiting for a property to be defined on an element
// callback is called synchronously immediately after the property is defined
if (!window.waitProperty) {
let objPropCallbacks = new Map();
window.waitProperty = function (obj, prop, callback) {
if (obj[prop] !== undefined) {
callback(obj[prop]);
return;
}
if (!objPropCallbacks.has(obj))
objPropCallbacks.set(obj, new Map());
let propCallbacks = objPropCallbacks.get(obj);
let callbacks;
if (!propCallbacks.has(prop)) {
propCallbacks.set(prop, []);
function runCallbacks(val) {
for (let callback of callbacks) {
callback(val);
}
}
let _val;
Object.defineProperty(obj, prop, {
get: () => _val,
set: function(val) {_val = val; delete obj[prop]; obj[prop] = val; runCallbacks(val); callbacks.length = 0;},
configurable: true,
enumerable: true
});
}
callbacks = propCallbacks.get(prop);
callbacks.push(callback);
}
}
///
/// Speed up jStorage by replacing it with a much faster version.
///
function jStorage(localOnly, debugOptions) {
'use strict';
const realWindow = window;
return function () { // IEF because we need to declare local window after accessing global
let window;
if (!debugOptions) {
debugOptions = {};
}
if (debugOptions.window) {
window = debugOptions.window;
} else {
window = realWindow;
}
const jStorage = {}; // the main jStorage object that exposes the API, shadows function name above
const callbacks = {}; // callbacks for event listeners
const backend = window.sessionStorage; // where the data ultimately gets stored
const name = 'sessionStorage';
const persistent = true; // whether backend is persistent
const cache = {}; // in memory read cache / complete representation of the stored data
let timeout = null; // timeout handle of timeout used to delete keys that have expired due to ttl
let soonestExpiration = Infinity; // soonest expiration time of any key
jStorage.version = 'lazyshallow-0.5.0';
jStorage.storageAvailable = function () {return persistent};
jStorage.currentBackend = function () {return name};
jStorage.storageSize = function () {
return JSON.stringify(jStorage._getOldFormat()).length;
};
jStorage.reInit = function () {
if (backend.jStorage) {
const obj = JSON.parse(backend.jStorage);
const expiration = obj.__jstorage_meta.TTL;
delete obj.__jstorage_meta;
for (let key in obj) {
if (obj.hasOwnProperty(key)) {
const exp = expiration[key];
cache[key] = [obj[key], exp ? exp : 0];
this._scheduleCleanup(key, exp, true);
// should not and does not fire observers
}
}
}
};
jStorage.get = function (key, def) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
if (typeof cache[key][0] === 'object' && !Object.isExtensible(cache[key][0])) {
cache[key][0] = deepcopy(cache[key][0]);
}
const val = cache[key][0];
return val === undefined ? null : val;
}
return typeof(def) == 'undefined' ? null : def;
};
jStorage.getTTL = function (key) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
const ttl = cache[key][1];
if (ttl) {
return ttl - (+new Date());
} else {
return 0;
}
}
return 0;
};
jStorage.deleteKey = function (key) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
delete cache[key];
this._fireObservers(key, 'deleted');
return true;
}
return false;
};
jStorage.flush = function () {
for (let key in cache) {
if (cache.hasOwnProperty(key)) {
this.deleteKey(key);
}
}
return true;
};
jStorage.index = function() {
return Object.getOwnPropertyNames(cache);
};
jStorage.listenKeyChange = function (key, callback) {
if (key !== '*') {
this._checkKey(key);
}
if (!callbacks[key]) {
callbacks[key] = [];
}
callbacks[key].push(callback);
};
jStorage.stopListening = function (key, callback) {
if (key !== '*') {
this._checkKey(key);
}
if (!callback) {
delete callbacks[key];
} else {
if (callbacks[key]) {
callbacks[key] = callbacks[key].filter(function (cb) { return cb !== callback });
}
}
};
jStorage.set = function (key, val, options) {
if (!options) {
options = {};
}
this._checkKey(key);
if (val === null || val === undefined) {
this.deleteKey(key);
return val;
} else {
cache[key] = [val, 0];
this.setTTL(key, options.TTL || 0);
this._fireObservers(key, 'updated');
return val;
}
};
jStorage.setTTL = function (key, ttl) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
if (ttl && ttl > 0 ) {
cache[key][1] = ttl + (+new Date());
} else {
cache[key][1] = 0;
}
this._scheduleCleanup(key, cache[key][1], false);
return true;
}
return false;
};
jStorage.noConflict = function(saveInGlobal) {
delete window.$.jStorage;
if (saveInGlobal) {
window.jStorage = this;
}
return this;
};
// dispatches event to listeners registered with listenKeyChange
jStorage._fireObservers = function (keys, type) {
keys = [].concat(keys || []);
for (let i=0; i<keys.length; i++) {
const key = keys[i];
const matchingCallbacks = (callbacks[key] || []).concat(callbacks['*'] || []);
for (let j=0; j<matchingCallbacks.length; j++) {
matchingCallbacks[j](key, type, cache[key]);
}
}
};
// verify key is a valid key
jStorage._checkKey = function (key) {
if (typeof key != 'string' && typeof key != 'number') {
throw new TypeError('Key name must be string or numeric');
} else if (key === '*') {
throw new TypeError('keyname/wildcard selector not allowed');
} else if (key === '__jStorage_meta') {
throw new TypeError('Reserved key name');
}
return true;
};
// schedules or reschedules the next TTL cleanup taking into account
// the given expiration timestamp
// If synchronous is true, the item will be cleaned up immediately if it
// has already expired as of the start of the call.
jStorage._scheduleCleanup = function(key, expiration, synchronous) {
if (expiration === 0) {
return;
} else if (synchronous && expiration < +new Date()) {
this.deleteKey(key);
} else if (expiration < soonestExpiration) {
soonestExpiration = expiration;
clearTimeout(timeout);
if (soonestExpiration < Infinity) {
setTimeout(_cleanupTTL, soonestExpiration - (+new Date()));
}
}
}
// delete everything with expired ttl and schedule this to be called
// again at the soonest expiration time of the remaining elements
jStorage._cleanupTTL = function () {
if (+new Date() > soonestExpiration) {
for (let key in cache) {
if (this.getTTL(key) < 0 && cache.hasOwnProperty(key)) {
this.deleteKey(key);
}
}
let newSoonest = Infinity
for (let key in cache) {
if (cache.hasOwnProperty(key) && cache[key][1] !== 0 && cache[key][1] < newSoonest) {
newSoonest = cache[key][1];
}
}
soonestExpiration = newSoonest;
}
if (soonestExpiration < Infinity) {
timeout = setTimeout(_cleanupTTL, soonestExpiration - (+new Date()) );
}
};
jStorage._getAll = function() {
const obj = {};
for (let key in cache) {
if (cache.hasOwnProperty(key)) {
obj[key] = cache[key][0];
}
}
return obj;
};
jStorage._getExpirations = function() {
const expirations = {};
for (let key in cache) {
if (cache[key][1] !== 0) {
expirations[key] = cache[key][1];
}
}
return expirations;
};
jStorage._getOldFormat = function () {
const oldFormat = Object.assign( jStorage._getAll(), {"__jstorage_meta":{"CRC32":{}}});
oldFormat.__jstorage_meta.TTL = jStorage._getExpirations();
return oldFormat;
}
const deepcopy = function (obj) {
const t = typeof obj;
if (obj === undefined || obj === null || t === 'string' || t === 'number' || t === 'boolean') {
return obj;
} else if (typeof obj === 'object' && obj.toJSON === undefined) {
if (Array.isArray(obj)) {
const len = obj.length;
const newArr = new Array(len);
for (let i = 0; i < len; i++) {
newArr[i] = deepcopy(obj[i]);
}
return newArr;
} else {
const newObj = {};
for (let key in obj) {
if (obj.hasOwnProperty(key)) {
newObj[key] = deepcopy(obj[key]);
}
}
return newObj;
}
}
throw 'bad deep copy';
};
if (!String.prototype.startsWith) {
Object.defineProperty(String.prototype, 'startsWith', {value: function(search, rawPos) {
const pos = rawPos > 0 ? rawPos|0 : 0;
return this.substring(pos, pos + search.length) === search;
}});
}
if (!Object.assign) {
Object.assign = function (target, source) {
for (let prop in source) {
if (source.hasOwnProperty(prop)) {
target[prop] = source[prop]
}
}
return target;
}
}
if (typeof JSTORAGE_DEBUG_EXTENSIONS !== 'undefined' ) {
JSTORAGE_DEBUG_EXTENSIONS(jStorage, debugOptions);
}
const _cleanupTTL = jStorage._cleanupTTL.bind(jStorage);
// save changes back in format old jStorage can read
window.document.addEventListener('visibilitychange', function() {
backend.jStorage = JSON.stringify(jStorage._getOldFormat());
});
jStorage.reInit();
if (!localOnly) {
realWindow.$.jStorage = jStorage;
}
// cleanup scope
if (!realWindow.JSTORAGE_DEBUG && realWindow.jStorage === jStorage) {
delete realWindow.jStorage;
}
return jStorage;
}();
}
///
/// Code for replacing jStorage after the old version has already loaded
///
function replace_jstorage() {
let observers = getObservers(true); // get copy of observers already registered to old jStorage and clear them from old jStorage
let js = jStorage(true);
// clear cache from old jStorage
$.jStorage.flush();
// replace existing reference to old jStorage with new jStorage
let oldJStorage = Object.assign({}, $.jStorage);
Object.assign($.jStorage, js);
// add event listeners to new jStorage
for (let key of Object.getOwnPropertyNames(observers)) {
for (let callback of observers[key]) {
js.listenKeyChange(key, callback);
}
}
$.jStorage = js;
}
// exploit fact that _observers.hasOwnProperty gets called during event firing for 'flushed' events
// by temporarily monkeypatching hasOwnProperty in the prototype of all objects
function getObservers(clear) {
let obj = {};
let oldF = obj.__proto__.hasOwnProperty;
let _observers;
$.jStorage.listenKeyChange('wk-lazy-jstorage-dummy', function () {});
$.jStorage.set('wk-lazy-jstorage-dummy', 1);
let oldData = sessionStorage.jStorage;
let observersCopy;
obj.__proto__.hasOwnProperty = function() {
_observers = this;
observersCopy = Object.assign({}, _observers);
// clear observers registered with old jStorage
for (let key in _observers)
if (oldF.call(_observers,key))
delete _observers[key];
return oldF.call(this);
};
$.jStorage.flush();
obj.__proto__.hasOwnProperty = oldF;
sessionStorage.jStorage = oldData;
delete observersCopy['wk-lazy-jstorage-dummy'];
$.jStorage.reInit();
$.jStorage.deleteKey('wk-lazy-jstorage-dummy');
if (!clear)
Object.assign(_observers, observersCopy);
return observersCopy;
}
main();
})();
Just the jStorage changes
/// Distributed under the Apache 2.0 License
///
/// Speed up jStorage by replacing it with a much faster version.
///
function jStorage(localOnly, debugOptions) {
'use strict';
const realWindow = window;
return function () { // IEF because we need to declare local window after accessing global
let window;
if (!debugOptions) {
debugOptions = {};
}
if (debugOptions.window) {
window = debugOptions.window;
} else {
window = realWindow;
}
const jStorage = {}; // the main jStorage object that exposes the API, shadows function name above
const callbacks = {}; // callbacks for event listeners
const backend = window.sessionStorage; // where the data ultimately gets stored
const name = 'sessionStorage';
const persistent = true; // whether backend is persistent
const cache = {}; // in memory read cache / complete representation of the stored data
let timeout = null; // timeout handle of timeout used to delete keys that have expired due to ttl
let soonestExpiration = Infinity; // soonest expiration time of any key
jStorage.version = 'lazyshallow-0.5.0';
jStorage.storageAvailable = function () {return persistent};
jStorage.currentBackend = function () {return name};
jStorage.storageSize = function () {
return JSON.stringify(jStorage._getOldFormat()).length;
};
jStorage.reInit = function () {
if (backend.jStorage) {
const obj = JSON.parse(backend.jStorage);
const expiration = obj.__jstorage_meta.TTL;
delete obj.__jstorage_meta;
for (let key in obj) {
if (obj.hasOwnProperty(key)) {
const exp = expiration[key];
cache[key] = [obj[key], exp ? exp : 0];
this._scheduleCleanup(key, exp, true);
// should not and does not fire observers
}
}
}
};
jStorage.get = function (key, def) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
if (typeof cache[key][0] === 'object' && !Object.isExtensible(cache[key][0])) {
cache[key][0] = deepcopy(cache[key][0]);
}
const val = cache[key][0];
return val === undefined ? null : val;
}
return typeof(def) == 'undefined' ? null : def;
};
jStorage.getTTL = function (key) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
const ttl = cache[key][1];
if (ttl) {
return ttl - (+new Date());
} else {
return 0;
}
}
return 0;
};
jStorage.deleteKey = function (key) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
delete cache[key];
this._fireObservers(key, 'deleted');
return true;
}
return false;
};
jStorage.flush = function () {
for (let key in cache) {
if (cache.hasOwnProperty(key)) {
this.deleteKey(key);
}
}
return true;
};
jStorage.index = function() {
return Object.getOwnPropertyNames(cache);
};
jStorage.listenKeyChange = function (key, callback) {
if (key !== '*') {
this._checkKey(key);
}
if (!callbacks[key]) {
callbacks[key] = [];
}
callbacks[key].push(callback);
};
jStorage.stopListening = function (key, callback) {
if (key !== '*') {
this._checkKey(key);
}
if (!callback) {
delete callbacks[key];
} else {
if (callbacks[key]) {
callbacks[key] = callbacks[key].filter(function (cb) { return cb !== callback });
}
}
};
jStorage.set = function (key, val, options) {
if (!options) {
options = {};
}
this._checkKey(key);
if (val === null || val === undefined) {
this.deleteKey(key);
return val;
} else {
cache[key] = [val, 0];
this.setTTL(key, options.TTL || 0);
this._fireObservers(key, 'updated');
return val;
}
};
jStorage.setTTL = function (key, ttl) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
if (ttl && ttl > 0 ) {
cache[key][1] = ttl + (+new Date());
} else {
cache[key][1] = 0;
}
this._scheduleCleanup(key, cache[key][1], false);
return true;
}
return false;
};
jStorage.noConflict = function(saveInGlobal) {
delete window.$.jStorage;
if (saveInGlobal) {
window.jStorage = this;
}
return this;
};
// dispatches event to listeners registered with listenKeyChange
jStorage._fireObservers = function (keys, type) {
keys = [].concat(keys || []);
for (let i=0; i<keys.length; i++) {
const key = keys[i];
const matchingCallbacks = (callbacks[key] || []).concat(callbacks['*'] || []);
for (let j=0; j<matchingCallbacks.length; j++) {
matchingCallbacks[j](key, type, cache[key]);
}
}
};
// verify key is a valid key
jStorage._checkKey = function (key) {
if (typeof key != 'string' && typeof key != 'number') {
throw new TypeError('Key name must be string or numeric');
} else if (key === '*') {
throw new TypeError('keyname/wildcard selector not allowed');
} else if (key === '__jStorage_meta') {
throw new TypeError('Reserved key name');
}
return true;
};
// schedules or reschedules the next TTL cleanup taking into account
// the given expiration timestamp
// If synchronous is true, the item will be cleaned up immediately if it
// has already expired as of the start of the call.
jStorage._scheduleCleanup = function(key, expiration, synchronous) {
if (expiration === 0) {
return;
} else if (synchronous && expiration < +new Date()) {
this.deleteKey(key);
} else if (expiration < soonestExpiration) {
soonestExpiration = expiration;
clearTimeout(timeout);
if (soonestExpiration < Infinity) {
setTimeout(_cleanupTTL, soonestExpiration - (+new Date()));
}
}
}
// delete everything with expired ttl and schedule this to be called
// again at the soonest expiration time of the remaining elements
jStorage._cleanupTTL = function () {
if (+new Date() > soonestExpiration) {
for (let key in cache) {
if (this.getTTL(key) < 0 && cache.hasOwnProperty(key)) {
this.deleteKey(key);
}
}
let newSoonest = Infinity
for (let key in cache) {
if (cache.hasOwnProperty(key) && cache[key][1] !== 0 && cache[key][1] < newSoonest) {
newSoonest = cache[key][1];
}
}
soonestExpiration = newSoonest;
}
if (soonestExpiration < Infinity) {
timeout = setTimeout(_cleanupTTL, soonestExpiration - (+new Date()) );
}
};
jStorage._getAll = function() {
const obj = {};
for (let key in cache) {
if (cache.hasOwnProperty(key)) {
obj[key] = cache[key][0];
}
}
return obj;
};
jStorage._getExpirations = function() {
const expirations = {};
for (let key in cache) {
if (cache[key][1] !== 0) {
expirations[key] = cache[key][1];
}
}
return expirations;
};
jStorage._getOldFormat = function () {
const oldFormat = Object.assign( jStorage._getAll(), {"__jstorage_meta":{"CRC32":{}}});
oldFormat.__jstorage_meta.TTL = jStorage._getExpirations();
return oldFormat;
}
if (!String.prototype.startsWith) {
Object.defineProperty(String.prototype, 'startsWith', {value: function(search, rawPos) {
const pos = rawPos > 0 ? rawPos|0 : 0;
return this.substring(pos, pos + search.length) === search;
}});
}
if (!Object.prototype.assign) {
Object.prototype.assign = function (target, source) {
for (let prop in source) {
if (source.hasOwnProperty(prop)) {
target[prop] = source[prop]
}
}
return target;
}
}
if (typeof JSTORAGE_DEBUG_EXTENSIONS !== 'undefined' ) {
JSTORAGE_DEBUG_EXTENSIONS(jStorage, debugOptions);
}
const _cleanupTTL = jStorage._cleanupTTL.bind(jStorage);
// save changes back in format old jStorage can read
window.document.addEventListener('visibilitychange', function() {
backend.jStorage = JSON.stringify(jStorage._getOldFormat());
});
jStorage.reInit();
if (!localOnly) {
realWindow.$.jStorage = jStorage;
}
// cleanup scope
if (!realWindow.JSTORAGE_DEBUG && realWindow.jStorage === jStorage) {
delete realWindow.jStorage;
}
return jStorage;
}();
}
const deepcopy = function (obj) {
const t = typeof obj;
if (obj === undefined || obj === null || t === 'string' || t === 'number' || t === 'boolean') {
return obj;
} else if (typeof obj === 'object' && obj.toJSON === undefined) {
if (Array.isArray(obj)) {
const len = obj.length;
const newArr = new Array(len);
for (let i = 0; i < len; i++) {
newArr[i] = deepcopy(obj[i]);
}
return newArr;
} else {
const newObj = {};
for (let key in obj) {
if (obj.hasOwnProperty(key)) {
newObj[key] = deepcopy(obj[key]);
}
}
return newObj;
}
}
throw 'bad deep copy';
};
jStorage();
Extensive QUnit Tests for jStorage changes
// Distributed under Apache 2.0 License
objAssign = function (target, source) {
for (let prop in source) {
if (source.hasOwnProperty(prop)) {
target[prop] = source[prop]
}
}
return target;
}
let stCalled = 0;
let cbCalled = 0;
let oldSetTimeout = window.setTimeout;
window.setTimeout = function(callback, t) {
stCalled++;
return oldSetTimeout(function() {cbCalled++; return callback();}, t);
};
function TimeoutsMonitor() {
this.reset = function() {
this.stCalledInitial = stCalled;
this.cbCalledInitial = cbCalled;
};
this.timeouts = function() {
return stCalled - this.stCalledInitial;
};
this.callbacks = function() {
return cbCalled - this.cbCalledInitial;
};
this.tlt = function(ub) {
return ok(this.timeouts() <= ub, '# calls to setTimeout: ' + this.timeouts());
};
this.cblt = function(ub) {
return ok(this.callbacks() <= ub, '# callbacks by setTimeout: ' + this.callbacks());
};
this.reset();
}
function getBasicWindow(window) {
return {JSON:window.JSON, addEventListener:window.addEventListener.bind(window), document:{addEventListener:window.document.addEventListener.bind(window.document)}};
}
function DummyStorage() {
Object.defineProperty(this, 'setItem', {enumerable: false, value: function (key, val) {this[key] = val} });
Object.defineProperty(this, 'getItem', {enumerable: false, value: function (key, val) {return this[key]} });
Object.defineProperty(this, 'removeItem', {enumerable: false, value: function (key, val) {delete this[key]} });
}
module( "test environment" );
test( "test runner is allowed to open popup windows", function () {
const w = window.open('about:blank');
if (!w) {
ok(false, 'you must configure your browser to allow popup windows in order for certain tests to successfully run');
} else {
ok(true);
w.close();
}
});
test( "sessionStorage exists in test environment", function () {
ok(sessionStorage);
});
module( "browser behavior does not violate assumptions" );
test( "sessionStorage is not bidirectionally synchronized across tabs", function() {
const a = window.open('about:blank');
const b = window.open('about:blank');
a.window.sessionStorage.test = 'a';
b.window.sessionStorage.test = 'b';
window.sessionStorage.test = '';
strictEqual(a.window.sessionStorage.test, 'a');
strictEqual(b.window.sessionStorage.test, 'b');
strictEqual(window.sessionStorage.test, '');
a.close();
b.close();
strictEqual(window.sessionStorage.test, '');
})
module( "basic / misc functionality" );
test( "flush/index", function() {
strictEqual($.jStorage.flush(), true);
deepEqual($.jStorage.index(), []);
$.jStorage.set("test", "value");
deepEqual($.jStorage.index(), ["test"]);
strictEqual($.jStorage.flush(), true);
deepEqual($.jStorage.index(), [])
$.jStorage.set('test2', null);
$.jStorage.set('test3', 'value');
$.jStorage.set('test3', null);
deepEqual($.jStorage.index(), []);
strictEqual($.jStorage.get("test"), null);
strictEqual($.jStorage.get("test2"), null);
strictEqual($.jStorage.get("test3"), null);
deepEqual($.jStorage.index(), []);
//let js = jStorage(true);
//deepEqual(js.index(), []);
$.jStorage.flush();
});
test( "_getAll", function () {
if ($.jStorage._getAll) {
$.jStorage.flush();
deepEqual($.jStorage._getAll(), {});
$.jStorage.set('testkey', 'value');
deepEqual($.jStorage._getAll(), {'testkey': 'value'});
$.jStorage.flush();
return;
}
expect(0);
});
test( "storageSize", function() {
$.jStorage.flush();
let size = $.jStorage.storageSize();
ok(size <= 50);
$.jStorage.set('k1', 'v1');
ok($.jStorage.storageSize() > size);
size = $.jStorage.storageSize();
$.jStorage.set('k1', 'v1');
equal($.jStorage.storageSize(), size);
$.jStorage.set('k2', 'v2');
ok($.jStorage.storageSize() > size);
$.jStorage.flush();
});
module( "set" );
test("undefined", function() {
strictEqual($.jStorage.set("test", undefined), undefined);
strictEqual($.jStorage.get("test"), null);
$.jStorage.flush();
});
test("zero", function() {
strictEqual($.jStorage.set("test", 0), 0);
strictEqual($.jStorage.get("test"), 0);
$.jStorage.flush();
});
test("missing", function() {
strictEqual($.jStorage.get("test"), null);
$.jStorage.flush();
});
test("use default", function() {
$.jStorage.set("value exists", "value");
strictEqual($.jStorage.get("no value", "def"), "def");
strictEqual($.jStorage.get("value exists", "def"), "value");
$.jStorage.flush();
});
test("string", function() {
strictEqual($.jStorage.set("test", "value"), "value");
strictEqual($.jStorage.get("test"), "value");
$.jStorage.flush();
});
test("boolean", function() {
strictEqual($.jStorage.set("test true", true), true);
strictEqual($.jStorage.get("test true"), true);
strictEqual($.jStorage.set("test false", false), false);
strictEqual($.jStorage.get("test false"), false);
$.jStorage.flush();
});
test("number", function() {
strictEqual($.jStorage.set("test", 10.01), 10.01);
strictEqual($.jStorage.get("test"), 10.01);
strictEqual($.jStorage.set(3, "value"), "value");
strictEqual($.jStorage.get(3), "value");
$.jStorage.flush();
});
test("object", function() {
let testObj = {arr:[1,2,3], 4:5, true:6, false:7, 8:true, 9:false, 10:null, 11:0, 0:12};
let original = {arr:[1,2,3], 4:5, true:6, false:7, 8:true, 9:false, 10:null, 11:0, 0:12};
deepEqual($.jStorage.set("test", testObj), original);
deepEqual(testObj, original);
propEqual($.jStorage.get("test"), original);
$.jStorage.flush();
});
module('delete');
test("deleteKey", function() {
deepEqual($.jStorage.index(), []);
$.jStorage.set("test", "value");
deepEqual($.jStorage.index(), ["test"]);
strictEqual($.jStorage.deleteKey("test"), true);
strictEqual($.jStorage.deleteKey("test"), false);
deepEqual($.jStorage.index(), []);
$.jStorage.flush();
});
module('ttl lifecycle');
asyncTest("TTL", function() {
expect(2);
$.jStorage.set("ttlkey", "value", {TTL:500});
setTimeout(function(){
equal($.jStorage.get("ttlkey"), "value");
setTimeout(function(){
strictEqual($.jStorage.get("ttlkey"), null);
$.jStorage.flush();
start();
}, 500);
}, 250);
});
asyncTest("setTTL", function() {
expect(2);
$.jStorage.set("ttlkey", "value");
$.jStorage.setTTL("ttlkey", 500);
setTimeout(function(){
equal($.jStorage.get("ttlkey"), "value");
setTimeout(function(){
strictEqual($.jStorage.get("ttlkey"), null);
$.jStorage.flush();
start();
}, 500);
}, 250);
});
asyncTest("setTTL - no expiration", function() {
$.jStorage.set("ttlkey", "value");
$.jStorage.setTTL("ttlkey", 0);
setTimeout(function(){
equal($.jStorage.get("ttlkey"), "value");
$.jStorage.flush();
start();
}, 100);
});
test("setTTL - negative ttl", function() {
$.jStorage.set("ttlkey", "value");
strictEqual($.jStorage.get("ttlkey"), "value");
$.jStorage.setTTL("ttlkey", -1);
strictEqual($.jStorage.getTTL("ttlkey"), 0);
strictEqual($.jStorage.get("ttlkey"), "value");
$.jStorage.flush();
});
asyncTest("getTTL", function() {
expect(3);
$.jStorage.set("ttlkey", "value", {TTL: 500});
setTimeout(function(){
ok($.jStorage.getTTL("ttlkey") > 0);
ok($.jStorage.getTTL("ttlkey") < 400);
setTimeout(function(){
strictEqual($.jStorage.getTTL("ttlkey"), 0);
$.jStorage.flush();
start();
}, 500);
}, 250);
});
asyncTest("getTTL - no expiration", function() {
$.jStorage.set("ttlkey", "value", {TTL: 0});
setTimeout(function(){
equal($.jStorage.getTTL("ttlkey"), 0);
$.jStorage.flush();
start();
}, 100);
});
asyncTest("increasing chain triggers okay", function() {
expect(10);
let tm = new TimeoutsMonitor();
$.jStorage.set("ttlkey1", "value", {TTL:500});
$.jStorage.set("ttlkey2", "value", {TTL:1000});
strictEqual($.jStorage.get("ttlkey1"), "value");
strictEqual($.jStorage.get("ttlkey2"), "value")
setTimeout(function(){
tm.cblt(5);
tm.tlt(5);
strictEqual($.jStorage.get("ttlkey1"), null);
strictEqual($.jStorage.get("ttlkey2"), "value");
setTimeout(function(){
tm.cblt(10);
tm.tlt(15);
strictEqual($.jStorage.get("ttlkey1"), null);
strictEqual($.jStorage.get("ttlkey2"), null);
$.jStorage.flush();
start();
}, 500);
}, 750);
});
asyncTest("decreasing chain triggers okay", function() {
expect(10);
let tm = new TimeoutsMonitor();
$.jStorage.set("ttlkey1", "value", {TTL:1000});
$.jStorage.set("ttlkey2", "value", {TTL:500});
strictEqual($.jStorage.get("ttlkey1"), "value");
strictEqual($.jStorage.get("ttlkey2"), "value")
setTimeout(function(){
tm.cblt(5);
tm.tlt(5);
strictEqual($.jStorage.get("ttlkey1"), "value");
strictEqual($.jStorage.get("ttlkey2"), null);
setTimeout(function(){
tm.cblt(10);
tm.tlt(15)
strictEqual($.jStorage.get("ttlkey1"), null);
strictEqual($.jStorage.get("ttlkey2"), null);
$.jStorage.flush();
start();
}, 500);
}, 750);
});
module("listeners");
asyncTest("stop listening - specific key", function() {
expect(1);
$.jStorage.listenKeyChange("testkey", function(key, action){
console.error(key,action);
$.jStorage.flush();
ok(false);
start();
});
$.jStorage.stopListening("testkey");
$.jStorage.set("testkey", "value");
setTimeout(function(){
ok(true);
$.jStorage.flush();
start();
}, 100);
});
test("specific key - updated", function() {
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
equal($.jStorage.get('testkey'), 'value');
equal(key, "testkey");
equal(action, "updated");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.set('testkey', 'value');
equal(called, 1);
$.jStorage.stopListening("testkey");
$.jStorage.flush();
});
test("any key - updated", function() {
let called = 0;
$.jStorage.listenKeyChange("*", function(key, action){
equal($.jStorage.get('testkey'), 'value');
equal(key, "testkey");
equal(action, "updated");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.set('testkey', 'value');
equal(called, 1);
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test("specific key - deleted", function() {
$.jStorage.set("testkey", "value");
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
equal($.jStorage.get('testkey'), null);
equal(key, "testkey");
equal(action, "deleted");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.deleteKey("testkey");
equal(called, 1);
$.jStorage.stopListening("testkey");
$.jStorage.flush();
});
test("any key - deleted", function() {
$.jStorage.set("testkey", "value");
let called = 0;
$.jStorage.listenKeyChange("*", function(key, action){
equal($.jStorage.get('testkey'), null);
equal(key, "testkey");
equal(action, "deleted");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.deleteKey("testkey");
equal(called, 1);
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test( "deleting nonexistant key" , function(){
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
called++;
});
$.jStorage.listenKeyChange("*", function(key, action){
called++;
});
$.jStorage.deleteKey("testkey");
equal(called, 0);
$.jStorage.stopListening("testkey");
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test( "setTTL" , function(){
$.jStorage.set("testkey", "value");
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
called++;
});
$.jStorage.listenKeyChange("*", function(key, action){
called++;
});
$.jStorage.setTTL("testkey", 500);
equal(called, 0);
$.jStorage.stopListening("testkey");
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test( "global/local listener order and flush calls listeners" , function(){
$.jStorage.set("testkey1", "value");
$.jStorage.set("testkey2", "value");
let called = [];
let f1 = function (key, action) { called.push([1, key, action]) };
let f2 = function (key, action) { called.push([2, key, action]) };
let f3 = function (key, action) { called.push([3, key, action]) };
$.jStorage.listenKeyChange("*", f3);
$.jStorage.listenKeyChange("testkey1", f1);
$.jStorage.listenKeyChange("testkey2", f2);
$.jStorage.flush();
deepEqual(called, [[1, 'testkey1', 'deleted'],[3, 'testkey1', 'deleted'], [2, 'testkey2', 'deleted'], [3, 'testkey2', 'deleted']]);
$.jStorage.stopListening("testkey1");
$.jStorage.stopListening("testkey2");
$.jStorage.stopListening("*");
$.jStorage.flush();
});
asyncTest( "scheduled deletion due to ttl expiration" , function(){
expect(4);
let timeout;
$.jStorage.set("testkey", "value", {TTL: 100});
$.jStorage.listenKeyChange("testkey", function(key, action){
strictEqual($.jStorage.get('testkey'), null);
strictEqual(key, 'testkey');
strictEqual(action, 'deleted');
$.jStorage.stopListening("testkey");
$.jStorage.flush();
clearTimeout(timeout);
start();
});
strictEqual($.jStorage.get('testkey'), 'value');
timeout = setTimeout(function() {
expect(2);
ok(false, 'timed out');
$.jStorage.flush();
start();
}, 350);
});
module("persistence");
asyncTest("page reload", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvalpagerel');
f.contentWindow.location.reload();
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvalpagerel');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page location replace", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvallocrep');
f.contentWindow.location.replace('about:blank');
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvallocrep');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page navigation - javascript", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvalnavjs');
f.contentWindow.location = 'about:blank';
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvalnavjs');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page navigation - user", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvalnavuser');
f.src = '';
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvalnavuser');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page close", function() {
const w = window.open('about:blank');
const storage = new DummyStorage();
let js = jStorage(true, {window:objAssign(getBasicWindow(w.window), {sessionStorage:storage})});
deepEqual(js.index(), []);
js.set('testkey', 'testvalclose');
console.log('closing');
w.close();
console.log('closed');
setTimeout(function() {
js = jStorage(true, {window:objAssign(getBasicWindow(window), {sessionStorage:storage})}); // window instead of w.window here because w.window can be null
console.log(storage);
strictEqual(js.get('testkey'), 'testvalclose');
js.flush();
start();
}, 250);
});
All in One HTML Document Containing jStorage changes, QUnit Tests, and QUnit Test Runner
<!DOCTYPE html>
<html>
<!-- This file is distributed under the Apache 2.0 License -->
<head>
<meta charset="utf-8">
<title>jStorage » QUnit test runner</title>
<link rel="stylesheet" href="http://code.jquery.com/qunit/qunit-1.14.0.css">
</head>
<body>
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<script src="http://code.jquery.com/qunit/qunit-1.14.0.js"></script>
<script>
if (!window.$) {
window.$ = {}
}
</script>
<script>
// New jStorage implementation
function jStorage(localOnly, debugOptions) {
'use strict';
const realWindow = window;
return function () { // IEF because we need to declare local window after accessing global
let window;
if (!debugOptions) {
debugOptions = {};
}
if (debugOptions.window) {
window = debugOptions.window;
} else {
window = realWindow;
}
const jStorage = {}; // the main jStorage object that exposes the API, shadows function name above
const callbacks = {}; // callbacks for event listeners
const backend = window.sessionStorage; // where the data ultimately gets stored
const name = 'sessionStorage';
const persistent = true; // whether backend is persistent
const cache = {}; // in memory read cache / complete representation of the stored data
let timeout = null; // timeout handle of timeout used to delete keys that have expired due to ttl
let soonestExpiration = Infinity; // soonest expiration time of any key
jStorage.version = 'lazyshallow-0.5.0';
jStorage.storageAvailable = function () {return persistent};
jStorage.currentBackend = function () {return name};
jStorage.storageSize = function () {
return JSON.stringify(jStorage._getOldFormat()).length;
};
jStorage.reInit = function () {
if (backend.jStorage) {
const obj = JSON.parse(backend.jStorage);
const expiration = obj.__jstorage_meta.TTL;
delete obj.__jstorage_meta;
for (let key in obj) {
if (obj.hasOwnProperty(key)) {
const exp = expiration[key];
cache[key] = [obj[key], exp ? exp : 0];
this._scheduleCleanup(key, exp, true);
// should not and does not fire observers
}
}
}
};
jStorage.get = function (key, def) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
if (typeof cache[key][0] === 'object' && !Object.isExtensible(cache[key][0])) {
cache[key][0] = deepcopy(cache[key][0]);
}
const val = cache[key][0];
return val === undefined ? null : val;
}
return typeof(def) == 'undefined' ? null : def;
};
jStorage.getTTL = function (key) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
const ttl = cache[key][1];
if (ttl) {
return ttl - (+new Date());
} else {
return 0;
}
}
return 0;
};
jStorage.deleteKey = function (key) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
delete cache[key];
this._fireObservers(key, 'deleted');
return true;
}
return false;
};
jStorage.flush = function () {
for (let key in cache) {
if (cache.hasOwnProperty(key)) {
this.deleteKey(key);
}
}
return true;
};
jStorage.index = function() {
return Object.getOwnPropertyNames(cache);
};
jStorage.listenKeyChange = function (key, callback) {
if (key !== '*') {
this._checkKey(key);
}
if (!callbacks[key]) {
callbacks[key] = [];
}
callbacks[key].push(callback);
};
jStorage.stopListening = function (key, callback) {
if (key !== '*') {
this._checkKey(key);
}
if (!callback) {
delete callbacks[key];
} else {
if (callbacks[key]) {
callbacks[key] = callbacks[key].filter(function (cb) { return cb !== callback });
}
}
};
jStorage.set = function (key, val, options) {
if (!options) {
options = {};
}
this._checkKey(key);
if (val === null || val === undefined) {
this.deleteKey(key);
return val;
} else {
cache[key] = [val, 0];
this.setTTL(key, options.TTL || 0);
this._fireObservers(key, 'updated');
return val;
}
};
jStorage.setTTL = function (key, ttl) {
this._checkKey(key);
if (cache.hasOwnProperty(key)) {
if (ttl && ttl > 0 ) {
cache[key][1] = ttl + (+new Date());
} else {
cache[key][1] = 0;
}
this._scheduleCleanup(key, cache[key][1], false);
return true;
}
return false;
};
jStorage.noConflict = function(saveInGlobal) {
delete window.$.jStorage;
if (saveInGlobal) {
window.jStorage = this;
}
return this;
};
// dispatches event to listeners registered with listenKeyChange
jStorage._fireObservers = function (keys, type) {
keys = [].concat(keys || []);
for (let i=0; i<keys.length; i++) {
const key = keys[i];
const matchingCallbacks = (callbacks[key] || []).concat(callbacks['*'] || []);
for (let j=0; j<matchingCallbacks.length; j++) {
matchingCallbacks[j](key, type, cache[key]);
}
}
};
// verify key is a valid key
jStorage._checkKey = function (key) {
if (typeof key != 'string' && typeof key != 'number') {
throw new TypeError('Key name must be string or numeric');
} else if (key === '*') {
throw new TypeError('keyname/wildcard selector not allowed');
} else if (key === '__jStorage_meta') {
throw new TypeError('Reserved key name');
}
return true;
};
// schedules or reschedules the next TTL cleanup taking into account
// the given expiration timestamp
// If synchronous is true, the item will be cleaned up immediately if it
// has already expired as of the start of the call.
jStorage._scheduleCleanup = function(key, expiration, synchronous) {
if (expiration === 0) {
return;
} else if (synchronous && expiration < +new Date()) {
this.deleteKey(key);
} else if (expiration < soonestExpiration) {
soonestExpiration = expiration;
clearTimeout(timeout);
if (soonestExpiration < Infinity) {
setTimeout(_cleanupTTL, soonestExpiration - (+new Date()));
}
}
}
// delete everything with expired ttl and schedule this to be called
// again at the soonest expiration time of the remaining elements
jStorage._cleanupTTL = function () {
if (+new Date() > soonestExpiration) {
for (let key in cache) {
if (this.getTTL(key) < 0 && cache.hasOwnProperty(key)) {
this.deleteKey(key);
}
}
let newSoonest = Infinity
for (let key in cache) {
if (cache.hasOwnProperty(key) && cache[key][1] !== 0 && cache[key][1] < newSoonest) {
newSoonest = cache[key][1];
}
}
soonestExpiration = newSoonest;
}
if (soonestExpiration < Infinity) {
timeout = setTimeout(_cleanupTTL, soonestExpiration - (+new Date()) );
}
};
jStorage._getAll = function() {
const obj = {};
for (let key in cache) {
if (cache.hasOwnProperty(key)) {
obj[key] = cache[key][0];
}
}
return obj;
};
jStorage._getExpirations = function() {
const expirations = {};
for (let key in cache) {
if (cache[key][1] !== 0) {
expirations[key] = cache[key][1];
}
}
return expirations;
};
jStorage._getOldFormat = function () {
const oldFormat = Object.assign( jStorage._getAll(), {"__jstorage_meta":{"CRC32":{}}});
oldFormat.__jstorage_meta.TTL = jStorage._getExpirations();
return oldFormat;
}
const deepcopy = function (obj) {
const t = typeof obj;
if (obj === undefined || obj === null || t === 'string' || t === 'number' || t === 'boolean') {
return obj;
} else if (typeof obj === 'object' && obj.toJSON === undefined) {
if (Array.isArray(obj)) {
const len = obj.length;
const newArr = new Array(len);
for (let i = 0; i < len; i++) {
newArr[i] = deepcopy(obj[i]);
}
return newArr;
} else {
const newObj = {};
for (let key in obj) {
if (obj.hasOwnProperty(key)) {
newObj[key] = deepcopy(obj[key]);
}
}
return newObj;
}
}
throw 'bad deep copy';
};
if (!String.prototype.startsWith) {
Object.defineProperty(String.prototype, 'startsWith', {value: function(search, rawPos) {
const pos = rawPos > 0 ? rawPos|0 : 0;
return this.substring(pos, pos + search.length) === search;
}});
}
if (!Object.prototype.assign) {
Object.prototype.assign = function (target, source) {
for (let prop in source) {
if (source.hasOwnProperty(prop)) {
target[prop] = source[prop]
}
}
return target;
}
}
if (typeof JSTORAGE_DEBUG_EXTENSIONS !== 'undefined' ) {
JSTORAGE_DEBUG_EXTENSIONS(jStorage, debugOptions);
}
const _cleanupTTL = jStorage._cleanupTTL.bind(jStorage);
// save changes back in format old jStorage can read
window.document.addEventListener('visibilitychange', function() {
backend.jStorage = JSON.stringify(jStorage._getOldFormat());
});
jStorage.reInit();
if (!localOnly) {
realWindow.$.jStorage = jStorage;
}
// cleanup scope
if (!realWindow.JSTORAGE_DEBUG && realWindow.jStorage === jStorage) {
delete realWindow.jStorage;
}
return jStorage;
}();
}
jStorage();
</script>
<script>
// QUnit Tests
objAssign = function (target, source) {
for (let prop in source) {
if (source.hasOwnProperty(prop)) {
target[prop] = source[prop]
}
}
return target;
}
let stCalled = 0;
let cbCalled = 0;
let oldSetTimeout = window.setTimeout;
window.setTimeout = function(callback, t) {
stCalled++;
return oldSetTimeout(function() {cbCalled++; return callback();}, t);
};
function TimeoutsMonitor() {
this.reset = function() {
this.stCalledInitial = stCalled;
this.cbCalledInitial = cbCalled;
};
this.timeouts = function() {
return stCalled - this.stCalledInitial;
};
this.callbacks = function() {
return cbCalled - this.cbCalledInitial;
};
this.tlt = function(ub) {
return ok(this.timeouts() <= ub, '# calls to setTimeout: ' + this.timeouts());
};
this.cblt = function(ub) {
return ok(this.callbacks() <= ub, '# callbacks by setTimeout: ' + this.callbacks());
};
this.reset();
}
function getBasicWindow(window) {
return {JSON:window.JSON, addEventListener:window.addEventListener.bind(window), document:{addEventListener:window.document.addEventListener.bind(window.document)}};
}
function DummyStorage() {
Object.defineProperty(this, 'setItem', {enumerable: false, value: function (key, val) {this[key] = val} });
Object.defineProperty(this, 'getItem', {enumerable: false, value: function (key, val) {return this[key]} });
Object.defineProperty(this, 'removeItem', {enumerable: false, value: function (key, val) {delete this[key]} });
}
module( "test environment" );
test( "test runner is allowed to open popup windows", function () {
const w = window.open('about:blank');
if (!w) {
ok(false, 'you must configure your browser to allow popup windows in order for certain tests to successfully run');
} else {
ok(true);
w.close();
}
});
test( "sessionStorage exists in test environment", function () {
ok(sessionStorage);
});
module( "browser behavior does not violate assumptions" );
test( "sessionStorage is not bidirectionally synchronized across tabs", function() {
const a = window.open('about:blank');
const b = window.open('about:blank');
a.window.sessionStorage.test = 'a';
b.window.sessionStorage.test = 'b';
window.sessionStorage.test = '';
strictEqual(a.window.sessionStorage.test, 'a');
strictEqual(b.window.sessionStorage.test, 'b');
strictEqual(window.sessionStorage.test, '');
a.close();
b.close();
strictEqual(window.sessionStorage.test, '');
})
module( "basic / misc functionality" );
test( "flush/index", function() {
strictEqual($.jStorage.flush(), true);
deepEqual($.jStorage.index(), []);
$.jStorage.set("test", "value");
deepEqual($.jStorage.index(), ["test"]);
strictEqual($.jStorage.flush(), true);
deepEqual($.jStorage.index(), [])
$.jStorage.set('test2', null);
$.jStorage.set('test3', 'value');
$.jStorage.set('test3', null);
deepEqual($.jStorage.index(), []);
strictEqual($.jStorage.get("test"), null);
strictEqual($.jStorage.get("test2"), null);
strictEqual($.jStorage.get("test3"), null);
deepEqual($.jStorage.index(), []);
//let js = jStorage(true);
//deepEqual(js.index(), []);
$.jStorage.flush();
});
test( "_getAll", function () {
if ($.jStorage._getAll) {
$.jStorage.flush();
deepEqual($.jStorage._getAll(), {});
$.jStorage.set('testkey', 'value');
deepEqual($.jStorage._getAll(), {'testkey': 'value'});
$.jStorage.flush();
return;
}
expect(0);
});
test( "storageSize", function() {
$.jStorage.flush();
let size = $.jStorage.storageSize();
ok(size <= 50);
$.jStorage.set('k1', 'v1');
ok($.jStorage.storageSize() > size);
size = $.jStorage.storageSize();
$.jStorage.set('k1', 'v1');
equal($.jStorage.storageSize(), size);
$.jStorage.set('k2', 'v2');
ok($.jStorage.storageSize() > size);
$.jStorage.flush();
});
module( "set" );
test("undefined", function() {
strictEqual($.jStorage.set("test", undefined), undefined);
strictEqual($.jStorage.get("test"), null);
$.jStorage.flush();
});
test("zero", function() {
strictEqual($.jStorage.set("test", 0), 0);
strictEqual($.jStorage.get("test"), 0);
$.jStorage.flush();
});
test("missing", function() {
strictEqual($.jStorage.get("test"), null);
$.jStorage.flush();
});
test("use default", function() {
$.jStorage.set("value exists", "value");
strictEqual($.jStorage.get("no value", "def"), "def");
strictEqual($.jStorage.get("value exists", "def"), "value");
$.jStorage.flush();
});
test("string", function() {
strictEqual($.jStorage.set("test", "value"), "value");
strictEqual($.jStorage.get("test"), "value");
$.jStorage.flush();
});
test("boolean", function() {
strictEqual($.jStorage.set("test true", true), true);
strictEqual($.jStorage.get("test true"), true);
strictEqual($.jStorage.set("test false", false), false);
strictEqual($.jStorage.get("test false"), false);
$.jStorage.flush();
});
test("number", function() {
strictEqual($.jStorage.set("test", 10.01), 10.01);
strictEqual($.jStorage.get("test"), 10.01);
strictEqual($.jStorage.set(3, "value"), "value");
strictEqual($.jStorage.get(3), "value");
$.jStorage.flush();
});
test("object", function() {
let testObj = {arr:[1,2,3], 4:5, true:6, false:7, 8:true, 9:false, 10:null, 11:0, 0:12};
let original = {arr:[1,2,3], 4:5, true:6, false:7, 8:true, 9:false, 10:null, 11:0, 0:12};
deepEqual($.jStorage.set("test", testObj), original);
deepEqual(testObj, original);
propEqual($.jStorage.get("test"), original);
$.jStorage.flush();
});
module('delete');
test("deleteKey", function() {
deepEqual($.jStorage.index(), []);
$.jStorage.set("test", "value");
deepEqual($.jStorage.index(), ["test"]);
strictEqual($.jStorage.deleteKey("test"), true);
strictEqual($.jStorage.deleteKey("test"), false);
deepEqual($.jStorage.index(), []);
$.jStorage.flush();
});
module('ttl lifecycle');
asyncTest("TTL", function() {
expect(2);
$.jStorage.set("ttlkey", "value", {TTL:500});
setTimeout(function(){
equal($.jStorage.get("ttlkey"), "value");
setTimeout(function(){
strictEqual($.jStorage.get("ttlkey"), null);
$.jStorage.flush();
start();
}, 500);
}, 250);
});
asyncTest("setTTL", function() {
expect(2);
$.jStorage.set("ttlkey", "value");
$.jStorage.setTTL("ttlkey", 500);
setTimeout(function(){
equal($.jStorage.get("ttlkey"), "value");
setTimeout(function(){
strictEqual($.jStorage.get("ttlkey"), null);
$.jStorage.flush();
start();
}, 500);
}, 250);
});
asyncTest("setTTL - no expiration", function() {
$.jStorage.set("ttlkey", "value");
$.jStorage.setTTL("ttlkey", 0);
setTimeout(function(){
equal($.jStorage.get("ttlkey"), "value");
$.jStorage.flush();
start();
}, 100);
});
test("setTTL - negative ttl", function() {
$.jStorage.set("ttlkey", "value");
strictEqual($.jStorage.get("ttlkey"), "value");
$.jStorage.setTTL("ttlkey", -1);
strictEqual($.jStorage.getTTL("ttlkey"), 0);
strictEqual($.jStorage.get("ttlkey"), "value");
$.jStorage.flush();
});
asyncTest("getTTL", function() {
expect(3);
$.jStorage.set("ttlkey", "value", {TTL: 500});
setTimeout(function(){
ok($.jStorage.getTTL("ttlkey") > 0);
ok($.jStorage.getTTL("ttlkey") < 400);
setTimeout(function(){
strictEqual($.jStorage.getTTL("ttlkey"), 0);
$.jStorage.flush();
start();
}, 500);
}, 250);
});
asyncTest("getTTL - no expiration", function() {
$.jStorage.set("ttlkey", "value", {TTL: 0});
setTimeout(function(){
equal($.jStorage.getTTL("ttlkey"), 0);
$.jStorage.flush();
start();
}, 100);
});
asyncTest("increasing chain triggers okay", function() {
expect(10);
let tm = new TimeoutsMonitor();
$.jStorage.set("ttlkey1", "value", {TTL:500});
$.jStorage.set("ttlkey2", "value", {TTL:1000});
strictEqual($.jStorage.get("ttlkey1"), "value");
strictEqual($.jStorage.get("ttlkey2"), "value")
setTimeout(function(){
tm.cblt(5);
tm.tlt(5);
strictEqual($.jStorage.get("ttlkey1"), null);
strictEqual($.jStorage.get("ttlkey2"), "value");
setTimeout(function(){
tm.cblt(10);
tm.tlt(15);
strictEqual($.jStorage.get("ttlkey1"), null);
strictEqual($.jStorage.get("ttlkey2"), null);
$.jStorage.flush();
start();
}, 500);
}, 750);
});
asyncTest("decreasing chain triggers okay", function() {
expect(10);
let tm = new TimeoutsMonitor();
$.jStorage.set("ttlkey1", "value", {TTL:1000});
$.jStorage.set("ttlkey2", "value", {TTL:500});
strictEqual($.jStorage.get("ttlkey1"), "value");
strictEqual($.jStorage.get("ttlkey2"), "value")
setTimeout(function(){
tm.cblt(5);
tm.tlt(5);
strictEqual($.jStorage.get("ttlkey1"), "value");
strictEqual($.jStorage.get("ttlkey2"), null);
setTimeout(function(){
tm.cblt(10);
tm.tlt(15)
strictEqual($.jStorage.get("ttlkey1"), null);
strictEqual($.jStorage.get("ttlkey2"), null);
$.jStorage.flush();
start();
}, 500);
}, 750);
});
module("listeners");
asyncTest("stop listening - specific key", function() {
expect(1);
$.jStorage.listenKeyChange("testkey", function(key, action){
console.error(key,action);
$.jStorage.flush();
ok(false);
start();
});
$.jStorage.stopListening("testkey");
$.jStorage.set("testkey", "value");
setTimeout(function(){
ok(true);
$.jStorage.flush();
start();
}, 100);
});
test("specific key - updated", function() {
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
equal($.jStorage.get('testkey'), 'value');
equal(key, "testkey");
equal(action, "updated");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.set('testkey', 'value');
equal(called, 1);
$.jStorage.stopListening("testkey");
$.jStorage.flush();
});
test("any key - updated", function() {
let called = 0;
$.jStorage.listenKeyChange("*", function(key, action){
equal($.jStorage.get('testkey'), 'value');
equal(key, "testkey");
equal(action, "updated");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.set('testkey', 'value');
equal(called, 1);
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test("specific key - deleted", function() {
$.jStorage.set("testkey", "value");
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
equal($.jStorage.get('testkey'), null);
equal(key, "testkey");
equal(action, "deleted");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.deleteKey("testkey");
equal(called, 1);
$.jStorage.stopListening("testkey");
$.jStorage.flush();
});
test("any key - deleted", function() {
$.jStorage.set("testkey", "value");
let called = 0;
$.jStorage.listenKeyChange("*", function(key, action){
equal($.jStorage.get('testkey'), null);
equal(key, "testkey");
equal(action, "deleted");
called++;
});
equal(called, 0); // listener not yet called
$.jStorage.deleteKey("testkey");
equal(called, 1);
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test( "deleting nonexistant key" , function(){
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
called++;
});
$.jStorage.listenKeyChange("*", function(key, action){
called++;
});
$.jStorage.deleteKey("testkey");
equal(called, 0);
$.jStorage.stopListening("testkey");
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test( "setTTL" , function(){
$.jStorage.set("testkey", "value");
let called = 0;
$.jStorage.listenKeyChange("testkey", function(key, action){
called++;
});
$.jStorage.listenKeyChange("*", function(key, action){
called++;
});
$.jStorage.setTTL("testkey", 500);
equal(called, 0);
$.jStorage.stopListening("testkey");
$.jStorage.stopListening("*");
$.jStorage.flush();
});
test( "global/local listener order and flush calls listeners" , function(){
$.jStorage.set("testkey1", "value");
$.jStorage.set("testkey2", "value");
let called = [];
let f1 = function (key, action) { called.push([1, key, action]) };
let f2 = function (key, action) { called.push([2, key, action]) };
let f3 = function (key, action) { called.push([3, key, action]) };
$.jStorage.listenKeyChange("*", f3);
$.jStorage.listenKeyChange("testkey1", f1);
$.jStorage.listenKeyChange("testkey2", f2);
$.jStorage.flush();
deepEqual(called, [[1, 'testkey1', 'deleted'],[3, 'testkey1', 'deleted'], [2, 'testkey2', 'deleted'], [3, 'testkey2', 'deleted']]);
$.jStorage.stopListening("testkey1");
$.jStorage.stopListening("testkey2");
$.jStorage.stopListening("*");
$.jStorage.flush();
});
asyncTest( "scheduled deletion due to ttl expiration" , function(){
expect(4);
let timeout;
$.jStorage.set("testkey", "value", {TTL: 100});
$.jStorage.listenKeyChange("testkey", function(key, action){
strictEqual($.jStorage.get('testkey'), null);
strictEqual(key, 'testkey');
strictEqual(action, 'deleted');
$.jStorage.stopListening("testkey");
$.jStorage.flush();
clearTimeout(timeout);
start();
});
strictEqual($.jStorage.get('testkey'), 'value');
timeout = setTimeout(function() {
expect(2);
ok(false, 'timed out');
$.jStorage.flush();
start();
}, 350);
});
module("persistence");
asyncTest("page reload", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvalpagerel');
f.contentWindow.location.reload();
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvalpagerel');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page location replace", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvallocrep');
f.contentWindow.location.replace('about:blank');
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvallocrep');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page navigation - javascript", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvalnavjs');
f.contentWindow.location = 'about:blank';
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvalnavjs');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page navigation - user", function() {
const f = document.createElement('iframe');
document.body.append(f);
let js = jStorage(true, {window:f.contentWindow});
deepEqual(js.index(), []);
js.set('testkey', 'testvalnavuser');
f.src = '';
setTimeout(function() {
js = jStorage(true, {window:f.contentWindow});
strictEqual(js.get('testkey'), 'testvalnavuser');
js.flush();
f.remove();
start();
}, 250);
});
asyncTest("page close", function() {
const w = window.open('about:blank');
const storage = new DummyStorage();
let js = jStorage(true, {window:objAssign(getBasicWindow(w.window), {sessionStorage:storage})});
deepEqual(js.index(), []);
js.set('testkey', 'testvalclose');
console.log('closing');
w.close();
console.log('closed');
setTimeout(function() {
js = jStorage(true, {window:objAssign(getBasicWindow(window), {sessionStorage:storage})}); // window instead of w.window here because w.window can be null
console.log(storage);
strictEqual(js.get('testkey'), 'testvalclose');
js.flush();
start();
}, 250);
});
</script>
</body>
</html>
[*] The implementation is designed to be 100% backwards compatible with the subset of features used and guarantees relied upon by WaniKani itself and common user scripts.
[**] Specifically, the time ends when the DOM finishes updating. Any tasks that happen after the DOM is finalized don’t count towards the measured time. This may result in an underestimate of the true latency of up to ~10ms.
[***] A deep copy is sometimes necessary during lessons, but only because WaniKani unecessarily freezes certain objects during lessons. (Freezing is also generally bad for performance.)
[****] To ensure a fair comparison, the change in the audio icon when auto playing audio in lessons was not included in measured latency.