Kaku - Japanese OCR Dictionary

Kaku: 画 (かく) - stroke (of a kanji, etc.), picture, drawing

Kaku is a fast, powerful Japanese dictionary that stays on top of all your apps. It uses optical character recognition (OCR) technology to recognize kanji on the device screen for you (rather than the slowww tedious process of looking up individual characters manually), making it perfect for Japanese learners who want to study by reading raw manga, play untranslated games, and so on without the hassle of switching apps.

https://kaku.fuwafuwa.ca/

1 Like

Hey, cool idea! Definitely continue working on it. What other features do you have planned?

You read my mind. I wanted something like this. For a while I was using https://play.google.com/store/apps/details?id=com.niftygnomes.popupjapanesedictionary on my phone but it was such a hassle to copy and paste things. Your version definite is more convenient.

Mempo said... Hey, cool idea! Definitely continue working on it. What other features do you have planned?
For now, my main priority is adding KANJIDIC2 to the database. After that, I'm going to be digging a bit deeper into Tesseract's library to include top OCR candidates to choose from by dragging the kanji around (as well as being able to edit manually in the case that the OCR is horribly off). This function seems to do what I need, so hopefully that won't be too bad:

/**
* Get a reading-order iterator to the results of LayoutAnalysis and/or
* Recognize. The returned iterator must be deleted after use.
*
* @return iterator to the results of LayoutAnalysis and/or Recognize
*/
public ResultIterator getResultIterator();


I also intend to hook into Android's clipboard events (just like popupjapanesedictionary) and provide a interface to search for entries manually. There also seems to be a thing called EPWING, but I honestly have no clue what that dictionary is (or where to even find it), so that's something for the future.

Epwing is a format to save dictionary data. There are different versions of this format with different feature sets. Recent versions also support multi-media. Many Japanese dictionaries are (or have been in the past) available for purchase in this digital format. They can be used by opening them in a reader application that supports Epwing dictionaries.

So it isn’t a dictionary itself. But supporting it as a format would allow people to add their favorite own dictionary that they purchased somewhere or created themselves.

I am not sure if those .idea directories should be in your git.

You know you don’t need to draw in Google Translate right? It has a rather good OCR.

Syphus said... You know you don't need to draw in Google Translate right? It has a rather good OCR.
I started work on this before Google started adding all their fancy contextual search/translation to Android and was definitely disappointed I didn't finish before them. I still think this would be useful, since Google Translate doesn't show kana or alternate meanings, but just translates.


Benkei-dono said... I am not sure if those .idea directories should be in your git.
 Yeah, I'm not sure either. I mostly followed https://intellij-support.jetbrains.com/hc/en-us/articles/206544839-How-to-manage-projects-under-Vers...
Xyresic said...I also intend to hook into Android's clipboard events (just like popupjapanesedictionary) and provide a interface to search for entries manually. There also seems to be a thing called EPWING, but I honestly have no clue what that dictionary is (or where to even find it), so that's something for the future.
jakobd said...

Epwing is a format to save dictionary data. There are different versions of this format with different feature sets. Recent versions also support multi-media. Many Japanese dictionaries are (or have been in the past) available for purchase in this digital format. They can be used by opening them in a reader application that supports Epwing dictionaries.

So it isn't a dictionary itself. But supporting it as a format would allow people to add their favorite own dictionary that they purchased somewhere or created themselves.
EPWING technical information is sort of hard to find, so maybe I can save you some time if you decide to head down that path.  It's technically just an Electronic Book format.

EB Library is a source code package that supports EPWING packages.
It's kind of difficult to find the latest source: http://green.ribbon.to/~ikazuhiro/dic/ebu.html
There's also a Debian Linux-related version (maybe it's identical, I don't know): https://launchpad.net/ubuntu/+source/eb/4.4.3-9

The documentation is in Japanese, but it also has code examples.  (Documentation is in the ./doc directory after you compile)
Many EPWING dictionaries are JIS X 0208 (character coding), so you may need to convert to Unicode for use with Android.
Xyresic said...
Syphus said... You know you don't need to draw in Google Translate right? It has a rather good OCR.
I started work on this before Google started adding all their fancy contextual search/translation to Android and was definitely disappointed I didn't finish before them. I still think this would be useful, since Google Translate doesn't show kana or alternate meanings, but just translates.


Benkei-dono said... I am not sure if those .idea directories should be in your git.
 Yeah, I'm not sure either. I mostly followed https://intellij-support.jetbrains.com/hc/en-us/articles/206544839-How-to-manage-projects-under-Vers...
 I'd recommend adding https://www.gitignore.io/api/android to your .gitignore since it stops all files that are not needed by android from being committed.

Please keep working on it if you can!! :slight_smile: An app like this was exactly what I was looking for!

rfindley said...The documentation is in Japanese...
 This sounds terrifying. I guess I'm going to hope I don't have to cross this bridge.

I was literally searching for the longest time two days ago for something like this, and I happened upon this post on my dashboard when I went to do my review… Man, this is almost too convenient.

Xyresicさん、ありがとうございました!
 
Also, I did steps 1-6, but it was having problems “obtaining the screenshot” or something like that. A full reboot seemed to do the trick for me. 

EDIT: It seems as if that issue occurs when I disconnect the “chromecast” connection that is enabled while Kaku is running. How does the system Chromecast function work in relation to Kaku? Does it have to do with how you can cast your device screen to a Chromecast device?

ichitaka said...EDIT: It seems as if that issue occurs when I disconnect the "chromecast" connection that is enabled while Kaku is running. How does the system Chromecast function work in relation to Kaku? Does it have to do with how you can cast your device screen to a Chromecast device?
This is because Kaku requires getting a screenshot of the screen programmatically, which Android only allows by its MediaProjection APIs added in Android 5. As far as I been able to test, there can only ever be one projection app active at the same time. Therefore any app that requires capturing the screen (i.e. Chromecast), would interfere with Kaku.

Ah, ok. That makes sense.

Just made a couple improvements:

  • Added KanjiDict
  • Show alternative words also deemed likely by the OCR engine by dragging the text
Demo: https://www.youtube.com/watch?v=Uq9AwxClY3g
APK in OP

I think the only major task left is figuring out how to deinflict the verbs now.

That looks really cool and useful, thanks.

I’m also jealous of your programming skill.

This could be really awesome.  I love how you can shape and move the box around anywhere on the screen for the OCR.  In the latest video example you had it over a spot with game dialogue.  This will be great for games and apps that have static text boxes, like RPGs, and could really revolutionize studying with other material.  

Another small update:

  • Add 2 more additional swipe actions: edit and delete (delete isn’t very optimized yet and has some bugs)
  • Add cropped word images to both swap and edit swipe actions



Also, if anyone has downloaded and installed the apk, I would love to know what your thoughts were! I’m sorely missing data on performance, especially how stable it is on some older 5.0 devices.

Good work there. I will try it later when I have some time.