Anyone else remember that time @viet thought V2 would be would be out of beta in 2017? 
The recent changes have been really good though. 
1 Like
I do remember X_X
3 Likes
Awesome, this is great
If this field is removed today it’ll break Tsurukame on iOS 
I’ve submitted an update to Apple, but as usual they’ve rejected it with no real reason. I’m going to keep trying but I’ve no idea when it’ll be accepted.
I’d appreciate it if you could hold off for a while on deploying this change.
And I know this is still an alpha API, but a bit more warning for breaking changes would be very helpful.
1 Like
We can hold off for a while — let us know how the review process with Apple goes, and we’ll drop the field when an update is out there. 
We’ll also work on more of a heads up for folks — userscripts can respond a bit more quickly than apps that face review processes…
The update was approved yesterday so it should be ok for you to drop the field now! Thanks so much for the extra couple of days!
Apple reviews are so inconsistent - I just resubmitted exactly the same binary and they approved it without any comments. 
@viet subject_ids param is prone to overflows (on every endpoint that accepts it) and causes error 500 if fed anything outside the int range (less than −32768 or more than 32767).
I can confirm that. Fortunately the largest subject ID is 8813, so a 16-bit ID is probably fine for now. Assignments and reviews, however are in the hundreds of millions. A 32-bit ID is fine for them right now, but I can see it breaking the limit in a few years.
Thanks for bringing this to our attention.
We are going to have to do better error handling on our end to make sure the values passed in subject_ids are within our DB column size limits.
EDIT: Just issued a PR.
A 2 byte column for the subject ID should be fine. I don’t think the app will ever exceed the limitation.
As for assignments and reviews… Assignment IDs are rolling on a 4 byte column. Just checked the the DB and we still have a very long way to go to reach 2_147_483_647.
The reviews are definitely on their way there though…
1 Like
The size is not the main point. It’s just the application should not crash when client accidentally misses a comma, issuing ?subject_ids=123456789 instead of ?subject_ids=12345,6789.
The issue stems from the ID value falling outside the supported range of its column on the database. When this happens the framework we use (Rails) is kind enough to raise an error to inform us one or more of the IDs falls outside the acceptable range (which I see in our log notifications the two attempts made). The problem is the app isn’t rescuing this error and handling it appropriately.
Edit:
After thinking about it I see your point. We could add a filter before the query to remove any out of range IDs.
Thanks for being patient with me while I finally understood the issue.
I’ve issued another PR which proactively addresses any id filter having any invalid values. The payload should still return a 200 code with the objects for the valid id values.
I’ll let you know when the PR has been merged in.
2 Likes
I’m not sure that’s the best way to handle it. In the event a client misses a comma like @atzkey said, then they need to be made aware that their caller is incorrect and needs to be fixed. It might be be better to catch the bad ids, and return a 400 error that says that the offending ID is invalid. If nothing else, it’ll prompt them to look at their request and hopefully see their problem.
I think that error suppression is a way to go in this case. It should definitely be an error 4xx for a direct /resource/:id request, but in case of filtering params not returning [some of] the requested ids is perfectly fine.
As in: “I am interested in /fruits, ?namely=remons,orenjis,merons. Please show me what you have. Ah, I see, no orenjis, じゃあね, no worries.”
{sigh} Good times with massive review processes.
Well, I’m glad it’s through and the update is in there! We’ll push out the code change on Monday (since we learned a long while ago that production pushes on Fridays mean that one test case you didn’t cover is, in fact, super important and breaks the entire site).
2 Likes
Aaaand it’s gone.
The PR was merged in a couple days ago. Should no longer be experiencing this issue.
1 Like
Awesome. Works like a charm.
Cache issue with Firefox and Edge
I am writing a userscript that uses the V2 API to access the user’s assignments and I am seeing what appears to be a cache invalidation issue with the V2 API. I experienced this issue in Firefox 63.0.1 and Edge 44.17763.1.0 on Windows 10. I have reproduced the issue with curl below.
My aim is to maintain the current list of assignments client-side by using the updated_after query parameter to avoid unnecessary requests.
1. Get updated assignments via V2 API (result as expected)
I get all the updated items since a particular timestamp.
$ curl -v 'https://api.wanikani.com/v2/assignments?updated_after=2018-11-10T12:50:56.896090Z' -H 'Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
* Trying 54.165.51.142...
[SSL negotiation removed]
> GET /v2/assignments?updated_after=2018-11-10T12:50:56.896090Z HTTP/1.1
> Host: api.wanikani.com
> User-Agent: curl/7.59.0
> Accept: */*
> Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
>
< HTTP/1.1 200 OK
< Server: Cowboy
< Date: Sat, 10 Nov 2018 14:23:59 GMT
< Connection: keep-alive
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: strict-origin-when-cross-origin
< Etag: W/"e1314f25cc40418730be8b6bf3264a2f"
< Last-Modified: Sat, 10 Nov 2018 13:40:38 GMT
< Content-Type: application/json; charset=utf-8
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: c0360687-d01e-4000-b5b8-72cbdd3f069a
< X-Runtime: 0.033765
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin,Accept-Encoding
< Transfer-Encoding: chunked
< Via: 1.1 vegur
<
* Connection #0 to host api.wanikani.com left intact
{"object":"collection","url":"https://api.wanikani.com/v2/assignments?updated_after=2018-11-10T12%3A50%3A56.896090Z","pages":{"per_page":500,"next_url":null,"previous_url":null},"total_count":1,"data_updated_at":"2018-11-10T13:40:38.571876Z","data":[{"id":2188975,"object":"assignment","url":"https://api.wanikani.com/v2/assignments/2188975","data_updated_at":"2018-11-10T13:40:38.571876Z","data":{"created_at":"2017-05-05T22:03:58.327141Z","subject_id":5955,"subject_type":"vocabulary","srs_stage":5,"srs_stage_name":"Guru I","unlocked_at":"2017-05-05T22:03:58.257652Z","started_at":"2017-05-05T22:03:58.257652Z","passed_at":"2017-05-18T20:39:31.642057Z","burned_at":null,"available_at":"2018-11-17T12:00:00.000000Z","resurrected_at":null,"passed":true,"resurrected":false,"hidden":false}}]}
2. Do the API request again with the ETag provided from step 1 (result as expected)
I get a 304 Not Modified response.
$ curl -v 'https://api.wanikani.com/v2/assignments?updated_after=2018-11-10T12:50:56.896090Z' -H 'Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' -H 'If-None-Match: W/"e1314f25cc40418730be8b6bf3264a2f"'
* Trying 52.207.5.158...
[SSL negotiation removed]
> GET /v2/assignments?updated_after=2018-11-10T12:50:56.896090Z HTTP/1.1
> Host: api.wanikani.com
> User-Agent: curl/7.59.0
> Accept: */*
> Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
> If-None-Match: W/"e1314f25cc40418730be8b6bf3264a2f"
>
< HTTP/1.1 304 Not Modified
< Server: Cowboy
< Date: Sat, 10 Nov 2018 14:26:31 GMT
< Content-Length: 0
< Connection: keep-alive
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: strict-origin-when-cross-origin
< Etag: W/"e1314f25cc40418730be8b6bf3264a2f"
< Last-Modified: Sat, 10 Nov 2018 13:40:38 GMT
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: ba4b6aa0-b88d-4900-9460-9c7da1ffbaf5
< X-Runtime: 0.026207
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< Via: 1.1 vegur
<
* Connection #0 to host api.wanikani.com left intact
3. Go to WaniKani — Log in and complete a review (result as expected)
I got the review wrong.
4. Do the API request again with the ETag provided from step 1 (result not as expected)
I get a 304 Not Modified response. I wasn’t expecting this because I had requested the assignments that had been updated since the given timestamp and I knew that there was a new assignment update since the last request.
$ curl -v 'https://api.wanikani.com/v2/assignments?updated_after=2018-11-10T12:50:56.896090Z' -H 'Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' -H 'If-None-Match: W/"e1314f25cc40418730be8b6bf3264a2f"'
* Trying 52.21.103.149...
[SSL negotiation removed]
> GET /v2/assignments?updated_after=2018-11-10T12:50:56.896090Z HTTP/1.1
> Host: api.wanikani.com
> User-Agent: curl/7.59.0
> Accept: */*
> Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
> If-None-Match: W/"e1314f25cc40418730be8b6bf3264a2f"
>
< HTTP/1.1 304 Not Modified
< Server: Cowboy
< Date: Sat, 10 Nov 2018 14:32:01 GMT
< Content-Length: 0
< Connection: keep-alive
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: strict-origin-when-cross-origin
< Etag: W/"e1314f25cc40418730be8b6bf3264a2f"
< Last-Modified: Sat, 10 Nov 2018 14:30:57 GMT
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: 99b7e3a9-e620-4ce5-9d18-6699b0946ed2
< X-Runtime: 0.033412
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< Via: 1.1 vegur
<
* Connection #0 to host api.wanikani.com left intact
5. Do the API request without the ETag (result not as expected)
When I do the request again, I get a different response body to the one in the first step because it includes the assignment that was updated in step 3. That part was expected but I didn’t expect the ETag to be the same. When I perform this request in Firefox, I get the same 304 response that I did with curl.
$ curl -v 'https://api.wanikani.com/v2/assignments?updated_after=2018-11-10T12:50:56.896090Z' -H 'Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
* Trying 35.173.6.94...
[SSL negotiation removed]
> GET /v2/assignments?updated_after=2018-11-10T12:50:56.896090Z HTTP/1.1
> Host: api.wanikani.com
> User-Agent: curl/7.59.0
> Accept: */*
> Authorization: Token token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
>
< HTTP/1.1 200 OK
< Server: Cowboy
< Date: Sat, 10 Nov 2018 14:38:42 GMT
< Connection: keep-alive
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: strict-origin-when-cross-origin
< Etag: W/"e1314f25cc40418730be8b6bf3264a2f"
< Last-Modified: Sat, 10 Nov 2018 14:30:57 GMT
< Content-Type: application/json; charset=utf-8
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: a86d9e73-bb49-4d4a-955d-05bc95ccc756
< X-Runtime: 0.036394
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin,Accept-Encoding
< Transfer-Encoding: chunked
< Via: 1.1 vegur
<
* Connection #0 to host api.wanikani.com left intact
{"object":"collection","url":"https://api.wanikani.com/v2/assignments?updated_after=2018-11-10T12%3A50%3A56.896090Z","pages":{"per_page":500,"next_url":null,"previous_url":null},"total_count":2,"data_updated_at":"2018-11-10T14:30:57.745927Z","data":[{"id":148634,"object":"assignment","url":"https://api.wanikani.com/v2/assignments/148634","data_updated_at":"2018-11-10T14:30:57.745927Z","data":{"created_at":"2017-04-12T12:25:40.419971Z","subject_id":5797,"subject_type":"vocabulary","srs_stage":5,"srs_stage_name":"Guru I","unlocked_at":"2017-04-12T12:25:40.419971Z","started_at":"2017-04-12T12:25:40.419971Z","passed_at":null,"burned_at":null,"available_at":"2018-11-17T13:00:00.000000Z","resurrected_at":null,"passed":true,"resurrected":false,"hidden":false}},{"id":2188975,"object":"assignment","url":"https://api.wanikani.com/v2/assignments/2188975","data_updated_at":"2018-11-10T13:40:38.571876Z","data":{"created_at":"2017-05-05T22:03:58.327141Z","subject_id":5955,"subject_type":"vocabulary","srs_stage":5,"srs_stage_name":"Guru I","unlocked_at":"2017-05-05T22:03:58.257652Z","started_at":"2017-05-05T22:03:58.257652Z","passed_at":"2017-05-18T20:39:31.642057Z","burned_at":null,"available_at":"2018-11-17T12:00:00.000000Z","resurrected_at":null,"passed":true,"resurrected":false,"hidden":false}}]}
1 Like