This is a heads up we’ll be putting WaniKani under maintenance mode on 2023-01-20T08:00:00Z→2023-01-20T12:00:00Z, up to four hours.
We are hoping the maintenance process will be done in less than two hours, given our results from performing a few dry runs. We are giving ourselves a buffer just in case the maintenance does not go as planned. During this period the website and the public API will be inaccessible.
We apologize for any inconvenience this may cause and thank you for your understanding.
We don’t take these long downtimes lightly and do our best to avoid these. Fortunately this type of downtime has not been a common occurrence for us. Unfortunately, this upcoming maintenance and resulting downtime is unavoidable.
The WaniKani Community boards will still be up and running during the downtime period. If you wish to participate in the community boards (for example, access the private boards or post), please ensure you are logged into the boards prior to the start of the maintenance period. Else, the public parts of the board is still accessible for your reading. The boards can be accessed at https://community.wanikani.com.
Again we apologize for any trouble this maintenance period may cause you.
I do my reviews almost every hour, I’ve got a quite strict pattern to avoid huge reviews piles per hour. With the fast levels I’m in it’s sometimes 400 reviews per day so I need to spread them out.
Lets say WK is down from 9am to 12am and I do reviews in Flaming Durtles at 9am, 10am and 11am. At 12am WK is back online.
When FD then syncs with WK, will it sync the hours I did the reviews at correctly or will WK then put all the reviews I did at 12am?
In my experience, reviews done on FD while WK is down or you have no network are synced with the actual time of completion.
WK’s trusts the API client to provide the correct time as the only one you would cheat by lying is yourself, which allows FD to work offline (I haven’t tested it but you should be able to go a full day without coverage and properly review your morning lesson at the same times you would have if the sync happened)
They will be delayed I believe, FD can’t tell WK that the reviews happened “in the past”, they’ll be taken into account only when sent to the server.
It’s mildly annoying for me as well because my radicals happen to reach guru just at the beginning of this window…
Are you sure? I have the opposite experience. I sync my progress at the end of the review sessions (because I like to undo like a scrub) and if I start a review session at the end of one hour and finish it at the beginning of the next, WK considers that all the reviews were done in the 2nd hour.
I think that FD uses the time you end each review session for all reviews in the batch if you activate “Lesson/review settings” > “Delay processing quizz results” to be able to undo as much as you want.
You could have smaller review batches (I like doing 10 at a time personally) to have more items validated during the first hour.
The API lets you set the completion time of the review. And this completion time will be used as the base for calculating the next review time using the SRS timing. The assignment.available_at is then updated with the calculated next review time.
There is some validation with the review completion time, if I remember correctly. Not an exhaustive list.
Completion time must be in the past
Completion time must be greater than the subject’s current assignment.available_at.
For example, the API will accept a review completed a week ago, so long as the POST body of the review meets the above criteria. And the next review time will be calculated using the “week ago” completed at timestamp as the base.
Based on the above example, if the SRS timing calculated is one day from base time, then the subject would immediately appear available to be reviewed again due to a week already having been past. The assignment.available_at will be timestamped with the value of base time plus one day.
I don’t know how FD submits reviews to the API or how FD’s offline mode works, so I can not comment on that.
If this was a more common occurrence we would spend resources on engineering a more controlled system with zero down-time option. But this event is something that occurs roughly every four years.
I did not expect Heroku, haha. (I love selfhosting)
But yes makes sense, but for future reference:
Provision a new replica DB (DB_A) with the old version of Postgres.
heroku maintenance:on
Change DB hosts’ IP addresses in “/etc/hosts” of online servers to use read-only replica DB (not DB_A). By this moment, all write operations will fail.
Run pg_upgrade (with “–link”) on DB_A to upgrade to the new version of Postgres, and promote DB_A to be a primary.
Replace all DB hosts’ IP addresses in “/etc/hosts” of online servers to use DB_A. By this moment, write operations would resume.
Re-provision new replica nodes with the new version of Postgres.
heroku maintenance:off
This above is just out my mind, please correct me on any mistakes.
By the way Heroku also have some docs for a follower switchover:
PS Of course dont switch the procedure now when you already did some dry runs without problems.
Our change over process follows what Heroku suggests.
The roadblock from having this done with zero downtime is their upgrade-in-place option. It requires stopping writes to the primary database, which means putting the app in maintenance mode while the upgrade process takes place on the follower.
Heroku’s Postgres upgrade process takes a while to finish (which is where all the downtime is happening) and when its executed on the follower, the follower detaches from the primary.
As far as I know Heroku doesn’t have a convenient process to be able to sync any new changes from the primary to the detached follower. So having the primary actively receiving new updates while the follower upgrades is not an available option.
There may be an option working with Heroku’s process to truly achieve zero downtime; we just haven’t figured it out yet.
But yeah… All boring details. Knock on wood the upgrade goes well and we won’t have to revisit this for another few years.
