Hey guys, I was gone for a while and just restarted my progress. I was on level 21 last September and kinda lost my streak after the whole API and app fiasco. A lot of other stuff happened in my life as well so I couldn’t keep it up without the help of the apps but those started failing + had no tracking anymore.
Funny enough I traveled to Japan last October lol
Anyway, since I restarted I really wanna know what the status is of the WaniKani API and if it’s usable for devs who are making mobile apps and scripts. The app I used on Ios was Jakeippu. I have no idea what things are working and not working.
What I need is the progress and streak tracking. I got a heatmap script that started failing around the time I stopped last year and it was too big of a motivator + planing tool to do WaniKani without it again. I would also want an app that has cross-tracking of progress.
If WaniKani is not able to do this anymore I might have to take some drastic measures.
I remember there were changes made to sessions or something where you could track what items you did when and if you did them correctly and when you would see them again and so on. I think one of the script devs mentioned they can’t access that anymore and have to store everything in cookies etc
I also remember that progress via mobile apps then did not affect the heat map because of this API change
I don’t think that’s possible at the moment for reasons @d-hermit has already identified.
I will note that there is a functioning heatmap userscript:
This only works if you use a single browser (and progress would disappear if you cleared your browser data), though, as it stores the review information locally. You wouldn’t be able to see this information on an app, and any reviews done on the app wouldn’t count as progress on the heatmap.
Yeah well, I am using that script already but without the fully functioning API it’s kinda useless if you use more than one device as you said. I would even pay a bit more for proper API access but the lack of communication and problem-solving from the dev side is pissing me off. I understand they probably have an unoptimized db query for the reviews that creates performance issues or a high amount of traffic. Unless they communicate what the actual problem is nobody can help them. Since there are so many devs in this community I hoped they would at least try!
Just tried the app I have been using and it just won’t start, sadly. it’s starting to frustrate me enough to the point where I am considering building my own shit. It would be a shit ton of work though ugh…
Depending on how far you want to go, you could deploy your own review database and write to it using a user script and/or a modified 3rd party app. Might be an interesting project to implement.
I’m doing another thing on the side, primarily to learn Go, but also hope to make useful tool to skip items I don’t need, like kana vocab, or stuff I know well.
Hey there, I’m the developer of Hakubun. Thanks for the shout out! @mersadajan, I’ve worked with the API a good amount, and it’s in a relatively stable state.
From my understanding, some older apps broke when kana vocabulary was added a while back, probably due to how the data was being pulled in those apps. I’m not too familiar with creating user scripts, but by nature they’re much more unstable since typically they’re not just relying on the API. But like some other people in the thread have said, it looks like there’s some existing options and (I believe?) a functioning endpoint to get all review statistics.
I’ve never used heatmaps myself, but may look into the idea in the future when Hakubun is in a more mature state
Well, the endpoint just gives you some general review statistics nothing you can use to generate a heatmap out of it. For heatmap generation you need to have access to all the reviews you ever did but they disabled that endpoint. I can imagine why this endpoint creates problems if it returns all reviews unpaginated at once and if every heatmap script pulls all reviews at every refresh! This is just a bad setup that could easily be fixed but as I understand it nobody wants to work on a solution for this as it is probably low prio for them
I would probably add pagination to the endpoint so that script creators don’t have to query all reviews at once all the time.
Ahh yeah, good point about the review stats endpoint. I’m curious how that userscript linked above deals w the heatmap, may look into the code if it’s open-source. It sounds like it’s caching a ton of data in local storage and that’s why it doesn’t work across browsers, but does the heat map history only begin when you first enable the userscript? Either way, to make it maintainable long-term you’d prob unfortunately have to create your own database and store the review data there, like you said earlier.
Yeah, I’d think that pagination would solve most of the reviews endpoint problems since pagination is enabled on most endpoints I’ve used. Though I’m guessing the main reason is that maintaining a database of that size gets spendy
Anyway, if you have any good ideas and are interested in contributing to open-source, maybe could add a heatmap to Hakubun eventually!
I think even before that it was saved in local storage + regular updates via API (maybe I am wrong) and later switched to just local storage. Can’t imagine the script dev waiting for the API response before displaying the data of thousands of reviews in the heat map. I think the API calls were used for background updates of the local storage. This is just speculation on my part though lol
I don’t think that the database costs get too high if you optimize your API services. The reviews are being saved in the database anyway and need to be used for all the “next review” calculations as well. Sooo… efficient querying of the db with pagination added to it should increase the cost too much. The amount of times that endpoint can be queried per day could also be limited as well. Would once per hour be ok? Man, the costs would be negligible…
I have barely any app dev experience, I am mostly a full-stack web dev, don’t know if i could contribute anything
Yeah true haha, I guess it also wouldn’t make sense to refetch data that you know wouldn’t change. That amount of data in local storage just seems wild to me, lol. Though I suppose it’s a common practice for a ton of web apps.
Fair point about the database costs, I was assuming that it’s not optimized well. But even if not, totally not defending it as a justification since there’s no way the wanikani subscriptions don’t easily cover that cost tenfold haha
Oh hey, I’m full-stack too! Though I lean much more frontend. Hakubun is actually written in React TypeScript and uses Capacitor to create the Android and iOS apps. Pros and cons to being non-native, but I’m not a fan of the other options available and have JavaScript/TypeScript experience. GitHub project is here if you ever wanna take a look
Technically they don’t even have to make all review data available for a heatmap to work, they could even just expose the summed up data grouped by day. While that would limit flexibility and dev freedom, it would at least offer a simple option to send out data. That data could even be cached or saved somewhere in that grouped form, so instead of sending out thousands of reviews, they’d only need to send one entry per day that contains the number of lessons and reviews and maybe some other details.
But as you rightly said, they have shown that this is not a priority for them, so one can only dream of progress in that direction.
For the time being I have limited my WaniKani use to one device so I can use the heatmap userscript. Before I used to do my reviews and lessons on multiple devices…
It’s stored in IndexedDB, which allows more storage than localStorage. The data is minimized, too. Each review is a an array containing the timestamp, the item ID, starting SRS, and number of incorrect meaning/reading answers, so it’s not too much data per item
