Those New Lessons in Your Queue

Yes this addresses it.

We started recording milestone events a few months back. A lot of this new data is being exposed in the new API we are working on. Slowly since implementing the recordings we have been updating the app to make use of these timestamps.

The root issue was, under certain conditions, not all records were getting their “unlocked_at” timestamp when they were suppose to. We identified the problem area, patched it, and sat on it a bit to make sure the problem didn’t persist. After that we had to figure out how to backdate the affected records.

Originally we used the record’s creation timestamp as the representative “unlocked_at”, but switched to using a different attribute to remove the dependency on the creation timestamp. The “unlocked_at” timestamp is the datetime when the item is first available in the lesson queue. We started to query for its presence to populate the lesson queue.

We had to identify records which were legitimately unlocked (the table is very large), but not timestamped, and then figure out the appropriate backdate to fill in for the timestamp.

Figuring out the backdate took a little work, but isn’t hard to figure out. It is the maximum datetime of the following:

  1. The maximum passed_at timestamp (when the item reached Guru in the srs stage) of the all its components. This is used if the item is at or below the user level at the time when the user passed all of the item’s components.
  2. The unlocked_at timestamp of the level progression data matching the item’s level. This one can get tricky if the user resets. This is for the case if the item in question is at a higher level than the user when they passed all of its components. It follows that the item was unlocked the moment the user reaches the level.

We wanted to be sure we were backdating the unlocked_at timestamp correctly.

Lots of details. But hey, hopefully someone learns from this.