Detecting page change in userscripts

[bfricka]

You didn’t get an error because you were shadowing the variable (redefining observer as the second argument from the callback). So you were just setting that local variable to null, effectively doing nothing.

[ctmf]

@LupoMikti

For my script, I check for the count of children on the new body because I noticed that the first time it observes the body the body is empty, the second time it is populated and that’s when I want to inject things.

Ok that’s the part that’s derailing my brain and making me keep putting “figure this out completely” off for ‘later’

if (m[0].target.childElementCount > 0) return;

seems to ignore if there ARE child nodes, and only end up running the setup function if the body IS empty. Is this essentially waiting for newBody to do whatever and however many changes it wants to do, and then only finally trigger when it becomes empty when it’s finally copied to the real page body? Sorry for asking for so much hand-holding, i’m (obviously) not a real-life programmer.

Edit:

@ctmf

but I guess hypothetically someone’s first navigation to wanikani might be some page other than the dashboard, and then my script wouldn’t run even when they navigated (via turbo) TO the dashboard. Interesting.

tested: confirmed. Which I guess is a somewhat good sign that I’m starting to get the concept

[LupoMikti]

It’s alright that’s a good question and one I’ve actually been trying to figure out passively myself since sharing it here. I didn’t leave any notes to myself so it’s been a struggle to retrace my steps.

What it is essentially saying in as plain English as I can muster right now is:

“If the new body has children after the mutation is detected, then we do not need to run the setup function and inject things because they should all already be there; if there are no children, then we need to do setup and specially handle the fact that we are coming from a turbo load”

Edit: and don’t worry about not being a “real life programmer”, neither am I! I very specifically do not want to work as a developer, but I find doing things like scripts or modifying some programs I use to my liking fun. Everything I’ve learned for javascript has been through helping out with and modifying the userscripts I use over the past year or two.

[ctmf]

JFC I can. not. make this work.

console.log(m);
if (m[0].target.childElementCount > 0) return;

never runs the script, all m’s are identical, always. if I comment out that line, I get two copies even after I check that it doesn’t exist before I inject. I’m tearing my hair out and about to go back to my terrible way that worked.

edit: they’re not identical, turns out, but I’m not sure how that’s helpful yet. They all have target.childElementCount > 0 though.

edit again, seems to work if I do that but make it > 1, instead of > 0. (aha, exactly because of that leftover svg element Sinyaven mentioned as I was typing this)

[Sinyaven]

@ctmf

Is this essentially waiting for newBody to do whatever and however many changes it wants to do, and then only finally trigger when it becomes empty when it’s finally copied to the real page body?

That is exactly what this code was doing. When WK first introduced Turbo loading and I tried to update ConfusionGuesser with as little effort as possible, I searched for a way to trigger my init() function only after Turbo has finished replacing the old page with the new page after a page navigation. After trying out all Turbo events and not finding one that fits my needs, I decided to use an undocumented (and therefore very brittle) behavior of Turbo: The turbo:before-render event provides the new body it is planning to use in e.detail.newBody. To find out when Turbo has finished replacing the old page with the new one, I use a MutationObserver to detect when the content of e.detail.newBody was removed, under the assumption that this happens exactly when Turbo places this content into the actual page body.

As said before, this is a very brittle method and has already broken a while ago, because the undocumented behavior has changed. Now, e.detail.newBody contains not only the div that will be moved to the actual page, but also an svg which for some reason stays behind in e.detail.newBody. My new hacky quick fix is to filter out this svg element:

// it seems like Turbo does not move the SVG element into document.body, so let's ignore it
function relevantRootElementChildren(rootElement) {
	return [...rootElement?.children ?? []].filter(c => c.tagName !== `svg`);
}

document.addEventListener("turbo:before-render", async e => {
	let observer = new MutationObserver(m => {
		if (relevantRootElementChildren(m[0].target).length > 0) return;
		observer.disconnect();
		observer = null;
		init();
	});
	observer.observe(e.detail.newBody, {childList: true});
});
init();

[Inserio]

@Sinyaven

I decided to use an undocumented (and therefore very brittle) behavior of Turbo: The turbo:before-render event provides the new body it is planning to use in e.detail.newBody.

Maybe this used to be undocumented?
But from that link you yourself posted (which I have had open in a separate tab, myself, since the latest breaking update), the turbo:before-render event explains:

Fires before rendering the page. Access the new <body> element with event.detail.newBody.

I realize this is of no real consequence, but I thought it worth mentioning. Or perhaps you were insinuating something else and I misread.

[ctmf]

By ‘undocumented’, I think Sinyaven meant it doesn’t say when newBody goes away or what’s guaranteed to be left behind in newBody. In fact the problem was the assumption that it would go empty when (at the exact time that) the page was rendered, and now it doesn’t.

[LupoMikti]

Ah, that’s what I get for being so behind on my reviews. The last time my code that checks > 0 works, the behavior hadn’t changed. I just checked, and indeed my observer no longer successfully injects. I’ll have to adapt what sinyaven has shared.

Interesting that sinyaven points out how brittle it is yet it is still the only method I’ve seen that has worked consistently whereas I run into more issues trying to hunt down the right turbo event(s).

[ctmf]

It would be nice if they helpfully included a turbo:page-is-ready event. But there isn’t one.

[Logograph]

You know, I happen to know this one when I made an OwO script for the POLL thread (don’t even ask)

You would use a MutationObserver object, an object that takes another object and observes its mutations and executes a callback

Okay that made no sense, let’s get to the nitty gritty

const observer  = new MutationObserver(muts => {
    muts.filter(m => m.attributeName === 'href') .forEach(m => {
        const newUrl = document.location.href;
        //do something with the new URL
    })
});

observer.observe(document.head, {
    attributes: true
});

1. You create a MutationObserver object that takes a callback processing all the changes from the DOM render lifecycle.
2. You use a filter method so that this object only cares for document.href mutations.
3. Use a forEach to achieve whatever you want to do.
4. Finally, with the observer instantiated, you point it where you want it to listen, in this case document.head. The object containing the attributes: true key-value pair is extremely important because it means the DOM will only trigger your observer when attributes change in document.head.

Well, I hope that made sense.

[LupoMikti]

Instead of the (likely much more expensive) array filter you should just use the attributeFilter property of the options parameter:

observer.observe(document.head, { attributeFilter: ['href'] });

[Logograph]

I thought of that, but then I wasn’t sure how many browsers support that

[LupoMikti]

Sorry, was pointing out compatibility but I’m gonna err on the side of caution and say there’s probably something I’m missing. Apologies for the quick-fire reply.

[Logograph]

No, it’s alright

You were right, it’s just that I first started out in this when browser compat was still something to worry about

[Inserio]

I’ve used some of the info in this thread to put out a library add-on to WKOF.

Wanikani Open Framework Turbo Events (greasyfork.org)

It’s literally something that I just threw together today, so feel free to offer feedback. I may end up putting it on Github in order to facilitate pull requests and other feedback if people find it useful or really want to contribute.

Also let me know if you’d rather it be listed as a regular script than a library so that it can be installed rather than only used via @require.

[rfindley]

@Inserio

I’ve used some of the info in this thread to put out a library add-on to WKOF.

Wanikani Open Framework Turbo Events (greasyfork.org)

If you make a forum thread for this, please tag me. I was planning to do something like this when I finally get done with my work project next month, so I’d like to follow along with your progress.

Sometime last year, I had started integrating a template for Turbo events into a few of my scripts:

Wanikani Open Framework [developer thread]

Thanks for the code link! I’ve published an update for wkof to fix the insert_script_link() specifically on the Lesson Quiz page. I had overlooked the fact that the quiz has a different base URL than the lessons. Second, I see you’re using wkof.on_page_event(). I don’t recommend using it since it was a quick-fix experiment solely to get Double-Check working quickly during the big WK change. It does work in very limited cases, but I will probably rewrite it or remove it someday. Instead, th…

I don’t know if there are any ideas in there that are still useful, but you’re welcome to look.

[LupoMikti]

I’ll let others weigh in on the library thing. I prefer non-library because I specifically don’t like it when the @requires aren’t up to date. By installing it myself (and making sure it is written in a way there are no conflicts between versions, all scripts that use it will use the latest version installed) I only ever have to update 1 script myself instead of waiting on the individual scripts to update the URLs.

But this is very helpful! I’ll mess around with it more later and see how it gets on. Just to be sure, this also has a hidden requirement that any script using it should @match https://www.wanikani.com/* instead of specific URLs, correct?

[Inserio]

@LupoMikti

Just to be sure, this also has a hidden requirement that any script using it should @match https://www.wanikani.com/* instead of specific URLs, correct?

Yes! I recently added that to a section of the readme but it slipped my mind to include it here as well.

[ctmf]

Thought I might update everyone who isn’t following the library thread. @prouleau seems to have figured out the “one weird trick” to make all this mutationObserver and pre-render stuff unnecessary. (at least, for my simple use case.)

A single setTimeout(). Brilliant, working flawlessly for me.

Summary:

(function () {
   // working functions { }

   // init function ( } 
   // checks we're on the right page, and there isn't already a
   //   copy of the inserted element before continuing
   //   this is where the wkof.ready.....then stuff ends up being

   // Make trigger for turbo reloads
   addEventListener("turbo:render", (e) => {
        setTimeout(() => init(), 0);
    });

   // run everything first time
   init();
})();

Turbo Events Library for Wanikani Open Framework

In my test I have found the turbo:render event works fine with setTimeout. A mutation observer is not needed in this scenario. function addTurboEventListener(){ addEventListener("turbo:render", (e) => { if (document.querySelector('.dashboard') !== null) { // Lets Turbo to complete its work in progress setTimeout(delayedExecution, 0); // delay execution to the end of the event queue }; function delayedExecution(){ …

[bfricka]

This may rankle the sensibilities of a few devs, but, even though I use MutationObserver extensively in my script, I still run a simple setInterval (currently running every 200ms) for a couple of ids on the page, and if the references change or become null, I deal with the lifecycle accordingly (tear down, and if they exist, reconstruct).

The reality is, if you have an element with and id that you can tie to the lifecycle of your script, this is the most foolproof way to ensure you have the DOM structure you need. getElementById is ridiculously fast (on a decent laptop, it will run at over 100M op/s).

I haven’t updated my script to work without refresh yet, but I should need to simply change the URL matcher.

That said, I probably wouldn’t use this technique with other selector APIs (like querySelector) unless I had no other choice. They tend to be fast too, but are orders of magnitude slower than getElementById, and I’m a bit particular about this sort of thing.