API changes - Get All Reviews

I would not mind if it was limited to 1,2,or 3 years.

2 Likes

I don’t know how recently is recently, but I can imagine some correlation between the last update removing the review summary page and a lot of people installing a script to replace it’s functionality, which I would assume needs to acces reviews in some way. I can’t say for sure that one uses the endpoint, but it seems plausible. I think it uses caching to not have to access the api on every pageload. It seems to no longer work for me in incognito mode, which could mean nothing, or that it will stop working as well once the cache invalidates.

7 Likes

Nvm, edited

I had the wrong link in there and changed it, but it doesn’t show as changed so that would be confusing if you already looked at it earlier. I think it is good now.

It doesn’t seem like this script is using the reviews endpoint at all. It’s only using the subjects endpoint, and I assume it does some magic with that, so I don’t think it should be broken.

Ah, that’s understandable. Seems like a tricky situation to overcome and I think the decision you made is of course for the better, at least for now.

Suppose you could limit the amount of API calls per user, or setup a secondary server for handling API calls which won’t bring down the rest of the site, if it’s busy. Obviously those aren’t quick fixes though, rather longer term goals. But whatever the solution, good luck!

It seems to use the “https://api.wanikani.com/v2/reviews” endpoint indirectly, trough the review_cache and wkof scripts. I think that is the one that got changed?

2 Likes

I had a quick look at it too and it looks like it is using localStorage. I can’t see any indication it uses the reviews endpoint.

@gijsc1 Just a heads up that we have had this issue for quite some time but you may be right that the recent changes may have resulted in a few more users using the stats sites and scripts.

The issue is at the database level and is due to the volume of data stored. Throwing extra servers at the problem wont fix it in this case I am afraid.

1 Like

Ah, ok, yes, lol, I looked in the header section, but for some reason missed the require that I was looking for even somehow.
Then it will be broken (@tofugu-scott)

1 Like

Would it be possible to make a new database for the reviews done in the past XXX amount of time with it’s own endpoint? That way it would never get overloaded by volume, and all other data can be solved with caching as reviews don’t change.

2 Likes

Thanks @Gorbit99 and great sleuthing @gijsc1. I have added that script to the list in the original post. It is quite a web that some of the scripts have, I can see why there is a domino affect when one breaks.

1 Like

While I am not surprised that such an API would cause performance problems, it is immensely disappointing to see it gone.

That’s more functionality gone, functionality that I used to keep motivated, to monitor my daily performance, to manage my workload, to explore how I was progressing, and even just to help me remember whether or not had completed my daily allotment of lessons.

3 Likes

Not at all. Thanks for the heads up (and for making the change that breaks scripts the least).

I’ve been nervous about this api for a while. As @Kumirei mentioned, it’s a LOT of data.

The ganbarometer tries to only retrieve new review data that isn’t cached in local storage, but caching is easy to screw up (I never make mistakes, of course :roll_eyes:, but users use different browsers and caches can be cleared). Even the cached data for just one user needs an uncomfortable amount of space.

That ganbarometer at least really never needs more than a few days worth of recent raw reviews. I think most scripts could deal with some sort of data reduction for older historical review data without needing the full, individual review structures for all time.

I’m looking forward to whatever you come up with, regardless.

This is using the POST /v2/reviews endpoint to add new reviews, which I presume is a very different endpoint from the GET /v2/reviews endpoint. I don’t expect this to cause any issues and indeed, Flaming Durtles appears to work normally for me.

cc @tofugu-scott

4 Likes

Yeah that’s what I said, sorry if it wasn’t clear.

1 Like

Would be great if you could find a solution that still allowed getting some reviews - maybe a much harsher rate limiter of reviews/min that can be fetched, or limiting the calls to 1yr of data at a time or something.
Totally understand that it must be a big burden on your servers but it would be very disappointing to lose the functionality of scripts such as the heatmap, ganbarometer, and Wanikani History website, considering WaniKani’s selection of built-in stats is virtually non-existent!

8 Likes

I am heartbroken! :broken_heart:
The streak on the Heatmap is the biggest motivator. My 11 year old and 8 year old kids check it everyday and hold me accountable.
Please don’t get me wrong - totally believe you when you say this decision was not taken lightly. If there is anyway Wanikani can allow me to track…some time in the future, it will be nice

19 Likes

Here a couple of the things that I noticed broke, that weren’t mentioned previously:

3 Likes

Heatmap FYI: the Lessons view should still work, so not all is lost

6 Likes

Aside from keeping count yourself, is there now no way to track how many reviews you’ve done each day?

4 Likes