LipSurf - WaniKani Voice Dictation. Review while you eat, do chores, etc


#1

I’ve been working on this beast for over a year. It’s voice control for Google Chrome and I’ve added a WaniKani plugin for it. So now you can dictate your reviews. It’s especially helpful for those with disabilities that affect their typing.

Update - As of 12/30/2018 the WaniKani plugin alone has essentially become a full time gig for me. The WaniKani features of this extension are in the “pro” version only to support it’s continued development. A version in the future may have just WK lessons free but reviews will remain PRO only.

Here’s a video guide:

Just a small part of the new features:

Works decently for testing your accent, too. Those ふ, る, and く’s can be really challenging for native English speakers especially!

General Usage

  1. Install the extension
    https://chrome.google.com/webstore/detail/lipsurf/lnnmjmalakahagblkkcnjkoaihlfglon

  2. (optional) install the lightning mode user script so you don’t have to say “next” or “次” all the time.

  3. Do the tutorial for LipSurf

  4. Go to the options and choose your primary English dialect (Australia, UK… etc.)

  5. Make sure LipSurf is activated (click to toggle it on/off from the chrome extension toolbar)

  6. Say “crocodile crab” or “wanikani” WHILE YOU ARE NOT ON A SPECIAL chrome://... page… IMPORTANT: you must be on a normal tab, like google.com or a LipSurf tab (options, tutorial etc.). This is a google-chrome extension API shortcoming.

  7. Speak your reviews in the appropriate language (Japanese for readings, otherwise some dialect of English)

    • Only the correct answer will get filled in… to prevent speech recognizer mis-hearings
    • Say “wrong” or “だめ” when you don’t know the answer (you have to manually mark any wrong answers)
    • (NEW in v1.8) click the live text (transcript) to make a correction or a custom synonym.
    • After 3 wrong answers (3 speech utterances) you will see the answer in the textbox, at this point you can decide whether you were right or wrong and say the appropriate command.
    • Say “correct” or “正しい” to manually mark a work as correct, if the speech recognizer is having trouble
    • (NEW in v0.5) say “えいご” to switch to English if in Japanese/reading mode and then you can spell out kana as you would on the keyboard. (eg. for しょうがつ just say s-h-o-u-g-a-t-s-u) and the answer will be inputted only if it’s correct. Super helpful for the short one-syllable kanji that recognition has trouble with, especially.
    • Say “info” or “じょうほう” to simulate clicking the info button for the current item
    • You can also spell out English words for meaning items
  8. Consider supporting the project! I really need support to keep the thing maintained and improving. I would also super appreciate reviews in the chrome webstore, if you can’t back the project in any other way but you think it’s cool.

Here’s the site that has more details:
https://www.lipsurf.com

Updates:
Most recent update: 1/12/2019 – v1.9.1 Bug fixes
See past updates here: https://www.lipsurf.com/updates/


The New And Improved List Of API and Third Party Apps
#2

What are you using for Japanese speech recognition?


#3

The html5 speech recognition API. Works quite well in Google Chrome. Doesn’t work in Firefox, yet!


#4

Looks great so far, but I got stuck when trying the help command. It doesn’t recognize close help any command anymore

edit: works again after turning it off and on again
edit2: I see now. You might want to mention this in the tutorial, if that’s what caused it

image


#5

So what’s going to be included in the “pro” version? I am intrigued.


#6

@Kumirei I don’t think the auto-shut-off was your issue, because I posted this less than 20m ago :smile:

Maybe there was some background noise, someone else speaking, or the internet being temporarily down? Either way, if anything is awry turning it off and on again is the best best, although I haven’t had to do that in months myself.

The pro version features are not in stone yet, but I’m thinking: wake word, input for textboxes, and “pro” plugins which would be more advanced plugins for gmail and such that I support directly – as opposed to community plugins which will always be free.

Thanks for being the first person to try it out on WK!


Wanikani + Speech Recognition =?
#7

Doing WaniKani so frequently and consistently is definitely exacerbating certain physical problems that I have, and has been causing me some worry. Thank you so much for thinking of those that struggle with physical limitations.

I’ll certainly be giving this a try! The garbage microphone on my computer will likely be a problem, but I’m excited to potentially have an alternative on days where studying conscientiously is actually the wrong choice for me, medically.

Thank you again for sharing your hard work!


#8

This is amazing!!! I’m going to try it out when I get home from work tonight. Thanks for making this.


#9

Oh, haha. I wasn’t keeping track. I don’t think any of those were they case, since it worked just fine up until I opened help and nothing changed in between.

Will most likely get pro then. I’ve tried other programs like this but they didn’t work as well as this seems to do. Especially not Window’s native assistants, haha.


#10

@neicul :eyes:


#11

@charliec364 @Helix


#12

What great timing! I’ve been thinking recently how much I would like to have something like this! I’m excited to try it out :slight_smile:


#13

Cool concept.

I tried it out. Unfortunately, I don’t think it is for me. I’m from northern england and it really STRUGGLES to understand me. I was saying next for few minutes (previous worked better). It be interesting to see if it would recognise Japanese words though i didn’t have anything to view.

I have a great clear dedicated mic also (Samson). I’ve always had trouble to Speech recognition (even with Google) - though Alexa seems to understand me ok (Amazon did great on that front).


#14

I was just chuckling to myself about something similar.

I’m a non-native English speaker raised on British English. It seems to be quite lost about what I’m saying. :rofl:

It can’t recognize me saying “WaniKani” at all, it seems. The things that rolled across my screen that it thought I said had me laughing out loud quite a bit.

I only had two reviews ready. It took quite a while to get it to recognize me saying 使う, but 高さ went fine.

Edit: with the Japanese recognition; I’m not bashing the recognition system. ^^ I’m not claiming my Japanese accent is good enough for software to recognize everything at once.


#15

Voice control while eating could be difficult :smiley:


#16

image

It said 0 would mean it didn’t turn off


#17

@Omun You guys need to change the language to English (UK). That should work much better for you. It should be on-par with google. The default is English (US).


#18

hmm OK, first bug. I’ll get that one taken care of in the next version. I think leaving some inactivity threshold on there is good though… you know it’s only for when you don’t say any valid command for that period.


#19

I can’t get it to recognize single syllable answers at all. Makes kanji reviews practically impossible.
Also can’t access help when it’s in Japanese. nevermind, of course you just have to say it in Japanese


#20

Some of the short utterances are really hard. Exaggerating them sometimes works. I would just say “ただしい” if I knew it. Sometimes I say three quick obviously wrong things first, to see the hint in the placeholder and then decide “正しい” or “だめ” Feel free to email me miko@lipsurf.com we can have a dialogue that way too.