Feature request: automatic OCR of captured images.

[IainB]

@mouser: Could you consider this please?
Based on this: Inside Microsoft OCR Libraries.

- I would really like to see if CHS could accommodate this:

...Perform OCR on any text in images as they are clipped ...

(i.e., similar to OneNote.)

- so that CHS would be able to do this with the captured images - i.e., just like with ordinary text capture clips:

...Look at this:
I have set up a child group in the CHS "tree" called "Auto-Tags". ...

Ideally, it might be most useful if the OCR'd text was attached somehow to the image file in the database, say to the CHS "Clip Text" part of the clip, so it would be searchable and copyable within CHS.
Or - just thinking aloud - this might (say) imply saving such images as .JPG files with the OCR'd text saved/appended as Alternative Text(?) or to (say) the Caption field in the IPTC section of that file. The idea would be to also enable things like Windows Search and image management tools (e.g., Picasa) to pick up the OCR'd text, though I am unsure whether that would even be possible with Windows Search without some kind of iFilter (e.g., as is required to index/search for text in .TIFF files).

[cranioscopical]

^ +1 

[kunkel321]

Hell yes, +1.  The only reason I keep SnagIt installed is for the "Text Grab" feature in v10. 
If CHS had OCR capabilities, it would blow that out of the water!

[IainB]

Thought I'd cross-post this relevant item from: Re: free ABBYY Screenshot Reader.

This relates to the above discussion and some separate discussions:
As a result of pursuing the idea of getting OCR data out of any text-containing images in my CHS database (per this request here: Feature request: automatic OCR of captured images.), I "Ducked" (DuckDuckgo) for things relevant to the subject, and happened upon this interesting post:

FREE OCR software: a survey of desktop and online tools - freewaregenius.com
Jun 18, 2013 By Priit 35 Comments
...
16. ABBYY Screenshot Reader
ABBYY Screenshot Reader is a screen capture software that can do screenshot OCR on the fly. Excellent recognition quality, amazing number of 160+ input languages can be selected, also multiple languages at a time. It can nicely handle data tables. ABBYY Screenshot Reader is reviewed here.
...
[attach]

[kunkel321]

I've used ABBYY lots of times.  It's decent.  I finally got myself a copy of Acrobat XI  which has built in OCR.  Otherwise I'd still be using the ABBYY one.  fyi the ABBYY people have a product called PDF Transformer.  Every once in a while it appears as a freebie on the internets.  Has the same engines as the screenshot reader, I'm sure.

[IainB]

Yes, the ABBYY software seems really rather good at what it does.
As described in EPSON Perfection V330 Photo Scanner + ABBYY and ArcSoft software, I first came across it in the bundled software that came with that scanner.
The last time I had Acrobat was in its version 7, but I don't use it now and currently get .PDF OCR processing via a FREE software - see PDF-XChange Viewer ($FREE version) - Mini-Review.

My thought with the ABBYY ScreenshotReader was that it might be worth exploring to see whether it could be incorporated into the CHS process somehow, to meet the requirement for automatic OCR of captured images (those captured by CHS). This could be (say) upon the capture of each individual image, or perhaps as a post-capture batch process, or something. I had effectively been doing the latter - albeit manually - using OneNote, but the OCR capability of ABBYY ScreenshotReader seems to be superior to OneNote's OCR capability.

Added note: By the way, this is not to forget the very relevant point that any images in .TIF/.TIFF format can be automatically  OCR'd for text and indexed/searched by Windows Desktop Search, if you have the .TIFF iFilter installed. In my view, for client-based databases, this in itself could be a good reason for duplicating text-bearing images into .TIFF format.
Similarly, I gather that any/most images - i.e., not just those in .TIFF formats - which are stored in the Evernote "cloud" are OCR'd and indexed for searching, and .PDF imaged documents stored in Google Drive can be OCR'd and the text searched/extracted.

[sphere]

+1 for this feature

I would also be interested in the OCR engine being hosted locally.  I came across a program that Screenotate that has open source roots.

I am not sure if I should start a new thread since this one is so old. Just wanted to chime in and say this is a feature that would interest me as well.

[mouser]

what i need is someone to find me a free/open source OCR utility, that i can add a nice little tool button to SC and CHS to call.

[Nod5]

There is always Tesseract
https://github.com/tesseract-ocr/tesseract
https://stackoverflow.com/a/26418458

Though I don't think there is a portable/"bundleable" Windows release of a recent Tesseract version. Only installers and they're many times the size of Clipboard Help+Spell.

[wraith808]

Unless Tesseract has become a lot better since the last time I evaluated it against Abbyy, I wouldn't suggest it.

[Lintalist]

I've had pretty good results with Tesseract actually - if you're going with the English 'fast' library data + the exe its only 7.8mb (other libraries are bigger and slower it seems but might produce better results). The tesseract.exe works just fine in 'portable' mode, no need to install it - just keep the exe + trainingdata file in the same folder. It can be slow if you feed it large image files but for screenshots it shouldn't take longer as a few seconds in my experience.

Of course the more languages you add the larger it gets and you may wish to offer some clear instructions and options to users how to add and handle them.

[Nod5]

if you're going with the English 'fast' library data + the exe its only 7.8mb

Could you link to that binary? Or are you building from source? The latest Windows binary setup packages for V5 alpha from https://github.com/UB-Mannheim/tesseract/wiki are 40MB in size. The .exe can be extracted with 7zip to 200MB of files. I don't know how many of those files are redundant when only using tesseract.exe for plain simple command line image to text conversion.

Example screenshot


Tesseract OCR result (took less than a second)

Code

T've had pretty good results with Tesseract actually - if you're going with the English ‘fast’ library data + the exe its only 7.8mb (other libraries are bigger and slower
it seems but might produce better results). The tesseract.exe works just fine in ‘portable’ mode, no need to install it - just keep the exe + trainingdata file in the
same folder. It can be slow if you feed it large image files but for screenshots it shouldn't take longer as a few seconds in my experience.

Of course the more languages you add the larger it gets and you may wish to offer some clear instructions and options to users how to add and handle them.

[wraith808]

Try tables and text on images, especially when the clarity is not there and the contrast is not great.  More complicated scenarios are where it started to fail for me.

[Nod5]

Try tables and text on images, especially when the clarity is not there and the contrast is not great.  More complicated scenarios are where it started to fail for me.

Might need a preprocessing step with binarize and other transforms on image in such cases. Yes more complicated.

[Lintalist]

@nod5 I use Vis2 - a AutoHotkey script to grab a screenarea which then does the OCR using Tesseract - if you look in the bin folder you'll see the exe and language files - https://github.com/iseahound/Vis2 - I also use that exe + language file(s) in batch files and other scripts to process image files and it works well enough.

Image on text is always going to be tricky - no doubt commercial software (which I also use) is better, I mainly use Vis2 to quickly grab a few paragraphs of text from locked PDFs which I can't be bothered to unlock or those pesky websites that find it a good idea to block select/copy using javascript. It's just a hotkey away and I have the text I need, simple & fast - and free

So the question is how many screenshots you are going to make with text on images or complex tables? My guess is not that many for most people. So even it works 50% of the time its worth a shot.

[wraith808]

Try tables and text on images, especially when the clarity is not there and the contrast is not great.  More complicated scenarios are where it started to fail for me.

Might need a preprocessing step with binarize and other transforms on image in such cases. Yes more complicated.

We don't have to do all of that with Abbyy.  It just picks them up.  And for the number of documents that we use yearly, that time saved processing and the simplification of the infrastructure is well worth it.  I've found that if you want the best in class, you have to pay for it.

[Lintalist]

Just fyi - you could, by the looks of it, add it relatively easily (short amount of code) for Windows 10 users by using the built-in OCR
"Windows.Media.Ocr: Provides optical character recognition (OCR) API for reading text from images." @ https://docs.microsoft.com/en-us/uwp/api/windows.media.ocr

AutoHotkey script(s) https://www.autohotkey.com/boards/viewtopic.php?f=6&t=72674