WaniKani currently in maintenance mode

I manually ran the service which handles the leveling up for your account. Looks like you leveled up. You should see the change now.

I am going to run this through all active users within the last 1.5 hours to make sure people who are eligible for level up get the level up.

Thanks for letting us know. And sorry about the issue.

Looks good now. Thanks for the support!

No worries about the downtime. Although I wonder, if you guys are based on the west coast, is there anything in place if WK goes down at a less favorable time/on the weekend or are you/other engineers always on call? :thinking:

not that it matters to me a ton, given I’ve been procrastinating a bunch of reviews for the past few hours

We have a rotation on PagerDuty, everyone’s favorite paging tool :sweat_smile:

Thanks for the heads up. Looks like our PaaS injects our page into an iframe (why?), which like you said may be causing the issue. I’ll take a look at it.

Thank you for taking care of things so quickly, Viet. You really work hard!

ありがとうございました!

Now it’s time to enjoy my coffee with my reviews.

Just want to bring some closure to this, for those interested in what happened.

The third-party dependency the application uses had some degradation going on, which the third-party provider automated system detected. The provider has an automated process to swap in a new instance to replace the degraded instance.

We have had this happened in the past and there has never been an issue. The swap happens in the background and is executed flawlessly without downtime.

However, for some reason we are still working to identify with the provider, there was an issue with the automated swapping of instances. This led to the issues we were seeing and which forced us to bring the app into maintenance mode to address the issue.

Again, really sorry for any problems this may cause.

Understandable, I didn’t notice this happened. Thank you for taking care of it so rapidly

incidentally, I can only imagine someone trying to do a perfect run in WK and suddenly ''boom* maintenance mode during a critical review hour.