Mokuro: Read Japanese manga with selectable text inside a browser

Japanese Language Resources
1

As I write this, 25 threads include mention of Mokuro. Of those, 16 mentions were by me, and one was an offshoot of one of my mentions.

I’ve considered adding Mokuro to the Ultimate Additional Japanese Resources List, but including Mokuro as part of a larger process leads to unlimited potential.

Note: I didn’t write Mokuro. Full credit goes to Maciej Budyś. I’m just a happy customer of this free software and use it every day.

Mokuro

History

History of OCR and Mokuro

Tools like Yomichan and Migaku can hook into text for quick look-ups. Although designed for use on web pages, in some circumstances, one can utilize such tools for digital novels, digital light novels, visual novels and video games, and video subtitles.

Manga has traditionally been excluded from this list because its text is embedded in image files.

Text in images can only be read through a process called optical character recognition (OCR). However, even Google’s Tesseract OCR has difficulty when given clear text images. Traditional OCR tools fail miserably when it comes to manga, with its various fonts, font sizes, small-resolution images for older releases, and other situations.

Enter a developer named Maciej Budyś. In January 2022, Budyś released code for a project called Manga OCR.

What makes this tool different from prior OCR applications? The software’s README document speaks for itself:

Between this application to OCR text, and an unrelated project called Comic Text Detector to isolate the text to run through OCR, I was able to start building a workflow to utilize manga in my Japanese learning journey better.

However, I didn’t have a good process for these tools. I needed a more cohesive tool.

Less than four months later, Budyś released a tool that does just that: Mokuro.

What does Mokuro do?

Mokuro takes in manga images and returns an HTML file. This HTML file allows one to navigate through the manga pages and select text. This allows using tools such as Yomichan and Migaku to look up words while reading digital manga.

image
image773×363 112 KB

image

Requirements

Python

Mokuro is written in Python, meaning it can be installed and run on any system that supports Python, including Linux, Windows, and Mac. In some cases, it can be installed via pip, Python’s package installer.

Digital Manga

This is the more difficult part because any manga sold digitally includes digital rights management (DRM), which prevents you from being able to run the manga images through Mokuro.

By using tools (that I probably can’t link to here), one can get around the DRM on the manga they bought from Amazon, Kobo, or Bookwalker. This allows accessing the images for one’s personal use.

Usage

Mokuro’s GitHub repository (linked at the top of this post) includes instructions on how to use the software.

Feeding a folder of manga pages into Mokuro results in an HTML file with selectable text.

Text Parser Dictionaries

On Chrome and other Chromium-based web browsers, text parsers such as Yomitan and Migaku won’t work with local Mokuro files until you enable the extension’s “Allow access to file URLs” option.

For me, the magic is in the potential.

This comes in two forms.

First, Mokuro stores its parsed information in JSON files. This allows developers like myself to write software that taps into and utilizes these files.

image
image791×382 148 KB

Second is the HTML file itself, which developers can utilize via Javascript.

image
image558×592 111 KB

I know I’m not the only one here using Mokuro. For this reason, I wanted a place where I could mention the various projects I’ve used Mokuro for. And, if there is interest, try to make them available for others to use.

Projects

I’ll post each project below and link to them from here.

If you have any Mokuro-based projects that are available for others to use, or that you would consider making available if there’s interest, post about them below and I’ll include them in the list.

46 Likes
3

Mokuro Sentences Extract

One “issue” with Mokuro is that it doesn’t know what order dialogue occurs on a page. This is something that cannot be resolved outside of complex AI that’s out of my league to write.

The next best thing: have a way to rapidly navigate through the pages of a manga and indicate the order of text boxes, then save a file containing the text in that order.

This project was prompted by a discussion that became this thread:

https://community.wanikani.com/t/x/60351/

The thread goes a bit deeper into utilizing the output when creating an auto-populated vocabulary spreadsheet, but that portion is a separate project.

Project code:

This code isn’t completely user-friendly, but if anyone had a practical use for it, there are things that I can do to improve it.

7 Likes
4

oh, sweet.

thank you for sharing, this is the highlight of my day for sure

1 Like
5

Mokuro Bookshelf

This one uses my modified version of Mokuro. [Note: I’ve moved this change to a separate branch, so my main fork mirrors the base Mokuro. I’ve updated the link here to point to the branch.]

It’s a single HTML file that displays all manga volumes that have been opened in (my modified version of) Mokuro. It organizes them into “Reading”, “Future”, and “Finished” sections.

image
image806×816 179 KB

This only works in Chromium-based browsers. Future releases of Chromium may not support it. Firefox does not support this.

Project code:

9 Likes
6

I love that color fill effect, what stops this from working on firefox?

1 Like
7

Manga Text Search

Update: I recommend using the Python implementation rather than this Ruby version.

This one is not currently available, but I can look into making it useable if there is interest. It’s currently tied to my directory structure and spans multiple programming languages.

The first part is a Ruby script that reads the JSON files from a volume folder, that were created by Mokuro.

This is the prominent function. 
def extract_text_with_image_names(manga_path, json_files)

    output_file = File.join(manga_path, "#{File.basename(manga_path)} Searchable Text.txt")
    return if File.file?(output_file)

    File.open(output_file, "w+") do |f|
        json_files.each do |file|
            image_file_name = File.basename(file, File.extname(file))
            parsed = JSON.parse(open(file).read, symbolize_names: true)
            parsed[:blocks].each do |block|
                f.puts("#{image_file_name}\t#{block[:lines].join}")
            end
        end
    end

end

This generates a single text file containing every line of dialogue extracted by Mokuro and the file name of the page the text is from.

The file is saved into the same folder as the JSON files, but I also have a separate process that copies it to a central location where all these text files are stored.

With the manga folders in a specified location, with series folders and volume subfolders, and with all the text files copied in a single folder, I use a Ruby file to search through them and generate an HTML file with my results:

search.rb 
require 'naturally'

# This needs to be manually changed to whatever I want to search for.
search = /何.{0,3}今の/

def output_match(match, manga_folder, search)
  image_filename, line_text = match.split("\t")
  image_file = "#{manga_folder}/#{image_filename}.jpg"
  image_file = "#{manga_folder}/#{image_filename}.jpeg" unless File.file?(image_file)
  unless File.file?(image_file)
    puts "Cannot find image file: #{image_file}"
    puts 'Maybe its extension is not .jpg or .jpeg?'
    return ''
  end
  line_text_with_html = line_text.gsub(/(#{search})/, '<strong>\1</strong>').chomp
  "<li tabindex='0' onfocus='showImage(this, \"#{image_file}\")'>#{image_filename}: #{line_text_with_html}</li>\n"
end

base_folder = '/home/chris/Documents/OCR/OCR Process/Outputs/OCR'

output = ''

output += '<link rel="stylesheet" href="styles.css">'
output += '<script src="script.js"></script>'

output += '<div id="parent">'
output += '<div id="matches">'

current_series = ''

files_to_check = Dir.glob("#{base_folder}/**/*.txt")
sorted_files_to_check = Naturally.sort(files_to_check)

sorted_files_to_check.each do |file_to_check|
  matches = []
  IO.foreach(file_to_check) do |line|
    matches.append(line) if line =~ search
  end
  next if matches.empty?

  # puts file_to_check
  series, volume = file_to_check.sub("#{base_folder}/", '').sub(' Searchable Text.txt', '').split('/')

  manga_folder = "/home/chris/Books/Comics/Japanese/#{series}/#{volume}"
  unless File.directory?(manga_folder)
    puts "Cannot find manga folder: #{manga_folder}"
    puts 'Implement checking other known locations.'
    next
  end

  if series != current_series
    output += "<h2>#{series}</h2>"
    current_series = series
  end
  output += "<h3>#{volume}</h3>"

  output += "<ul>\n"
  matches.each do |match|
    output += output_match(match, manga_folder, search)
  end
  output += "</ul>\n"
end
output += '</div>'

output += '<div id="page">'
output += '<a id="link" target="_blank"><img id="image" /></a>'
output += '</div>'

output += '</div>'

File.write('results.html', output)

A couple of files accompany the saved file:

script.js 
var lastCopiedText = '';

function showImage(element, imagePath) {
	console.log(`ShowImage: ${imagePath}`)
	document.getElementById("link").href = imagePath;
	document.getElementById("image").src = imagePath;
	if (lastCopiedText != imagePath) {
		navigator.clipboard.writeText(imagePath);
		lastCopiedText = imagePath;
	}
}
style.css 
a {text-decoration: none; color: black;}

h2 {
position: sticky;
background-color: white;
top: 0;
}

#parent {
display: flex;
justify-content: center;
height: 99%;
}

#matches {
width: 600px;
height: 99%;
overflow: auto;
}

#page {
width: 600px;
height: 99%;
}

ul {
list-style: none;
padding-left: 0;
}
li {
border: solid thin beige;
padding: 1px 2px;
cursor: pointer;
}
li:hover { background: lightblue; }
li:focus { background: pink; }

#image {width: inherit; position: sticky; max-height: 100%;}

strong {color: #ff003c; font-weight: normal;}

With those, I can change the search term in the Ruby file and re-run it. Then, open the generated HTML file in a web browser.

This gives a page where I can easily view manga pages (from my collection) that contain what I’m looking for:

image
image1219×868 138 KB

(For anyone wondering how I can pull examples of any random vocabulary or grammar from several manga at a moment’s notice in book clubs…this is it.)

9 Likes
8

That fill effect was so difficult to get working right! My initial solution resulted in the images showing with bad quality and no way to fix it via CSS.

Mokuro stores information in the browser’s “local storage”.

In Chrome, every page opened in file:// has full access to all file:// local storage.

This allows me to create an HTML file that when opened in file:// can access all the Mokuro files’ local storage data.

In Firefox, a file:// page can only access its own local storage and cannot access the local storage from other file:// pages.

This means my page cannot view any of the Mokuro files’ local storages.

Accessing that local storage is vital because Mokuro stores your progress in it. Further, my fork of Mokuro stores information about which image is the cover image, so my page can display that.

If anyone knows how to “work around” this in Firefox, I’d love to know (if it’s possible).

4 Likes
9

I daily drive firefox (now that chromium basically broke for me and broke me in turn). I’ll try to look into it if I get some free time

2 Likes
10

Thank you so much this! Looks complicated or at least a bit more effort than I can give atm, so I’ll have to give this a proper read tomorrow.

1 Like
11

Ah, there it is :smiley:
Setting up the thread to Watching!

1 Like
12

Ah, so you can search using regex, pretty cool! I was wondering how you could hit those different results, make sense!

Editing my post so I don’t have 3 replies in a row: @ChristopherFritz , can we also use this thread to prompt you for lookups? :sweat_smile: I’d love to see if and how 面倒臭い is used in the wild in the Mangas you have
(Will look at installing Mokuro myself later today, maybe I’ll have some more technical questions then!)

1 Like
13

It probably wouldn’t be practical in the chance of getting too many requests.

But for this one, at least…

シャドーハウス カラー版 4

08_891461_891184_2_148_001: ．．．正直ジョンは私的には面倒臭いけど．．．

image

シャドーハウス カラー版 7

08_891819_891184_2_101_001: 私が任命されたらどうしようっ面倒臭そうだしやだなっ

image

Those are my only two results when the kanji 臭 is used.

However, when that part is kana...
3 Likes
14

Thanks a lot :smiley:

15

So I’m trying to install it right now but I’m running into some issues because this is quite foreign to me. I’m on this part:

mk
mk1518×220 8.67 KB

I’m not sure how to run it? I copied it into my command prompt, but nothing happens. I don’t know what I’m doing wrong, I’m fairly certain I’ve installed python.

1 Like
16

Just to be sure, this is the wrong place to put it:

image

(The screenshot is from Windows, but if you are on a different operating system that has a similar command prompt, you can probably figure what I’m referring to.)

This is the right place to put it:

image

If you’re on Windows, and you’re on the first item above, type “cmd” (without quotes) to get to the command window in the second screenshot above.

Now, if you were already in the right place, or if you’re just getting there now, you should be able to use the following commands to see what’s on your system:

python3

chris@mercurius:~> python3
Python 3.10.9 (main, Dec 08 2022, 14:49:06) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.

(My system is currently set to default to Python 2, so I typed “python3” to ensure I’m running the right version.)

Bad response: A message saying python3 wasn’t found.

Good response: What I pasted above. (This puts you into an instance of Python. Press Ctrl+D to edit out of it.)

pip3

chris@mercurius:~> pip3

Usage:   
  pip3 <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.

Liksewise: Bad response is a message about the program not being found. Good response is the above (which I have just a portion of above).

If you have Python3 and Pip3, you’re good to go. You can run the “pip3 install mokuro” command.

If you are missing either, you can install them the Python website:

2 Likes
17

What about all kana? For science.

2 Likes
18

You might be able to run it like
python3 pip …

1 Like
19

No results. I even tried searching allowing for both hiragana and katakana. I guess Yotsuba never says it?

2 Likes
20

I don’t remember her doing so, but did you try both めんど and めんどうくさい?

2 Likes
21

Today I learned it’s めんど, not めんどう.

Well, how about that.
3 Likes