hard: find a word on a window, highlight it

[urlwolf]

This might be a hard problem: locate a word on the screen and highlight it. In ANY type of window (adobe acrobat, a browser, MS word...).

This problem has three parts
1 - getting a reliable text dump
A 'hacky' approach would be to click on the area that displays text, send ctrl a + ctrl c, put that into the clipboard and the operate from there. This is application-general. Nice. A (maybe) cleaner approach would be to write rules for each application to save a text dump (e.g., sending the kb shortcuts for 'save as', 'export' etc depending on each application. It'd generate spaghetti code , not very elegant .

I can see that AutoIT windows spy drops two interesting pieces of information under:
>>>>>>>>>>>( Visible Window Text )<<<<<<<<<<<

>>>>>>>>>>>( Hidden Window Text )<<<<<<<<<<<
However this is most of the time uninteresting information such as the name of the controls, buttons, etc, nowhere near the full text dump that I'm thinking about.

A similar approach -anything out there that does like autoIT spy but grabs all text in a window?- would be an elegant solution...
2 - finding the word in the text dump
Here one may need some interesting algorithms for cases where the word appears more than once. To simplify, pick the first occurence.

3- locate the word in the native application! (hard!)
The only application-general way of doing this to do an OCR of a screen-capture of the text area, and then use some proprietary (all decent OCR libraries are proprietary ) function call to find the coordinates of that word. then draw a circle on top to highlight it...

If this was application-specific (e.g., firefox only, maybe one can highlight the word using UML/javascript (the language for FF extensions) to highlight the word.

Am I overcomplicating things? Do you have any tricks/hacks to solve this problem?

Thanks!
(PS: I have posted this in the ahk forum, as I'm using ahk to quickly get a mockup working...)

[f0dder]

It's not going to be an easy task, for the exact same reason that doing "scrolling screen capture" is so damn hard - various applications handle things differently, even when there's no good reason to do so. THEORETICALLY you should be able to simply send a WM_GETTEXT window message to toplevel windows, but that won't work for formatting etc... and even if this would work, getting the screen coords of the text is another bother.

[mouser]

Yeah, doing this reliably for different kinds of applications may be very difficult.  You might be able to do well on a few specific applications though, and have moderate success in general, but it's going to be a bit painful.

ps. This should probably be in developer's corner.. I'll move it there in a bit.

[rjbull]

I'm probably missing something, but FWIW:

If you just want to highlight a word, try Fanix Software's Hilitext.

If you want to extract the highlit text, Structurise's Kleptomania.  Freeware solutions that get some of the information include Nir Sofer's SysExp, Peter Boulton's WinScraper with alternate direct download link http://www.dataper.d.../WinScraperSetup.exe, and Robin Keir's ShoWin, now owned by Foundstone (link from Robin Keir's page).  Of these, WinScraper and ShoWin both behave rather like AutoIt's WindowSpy or PowerPro's WindowInfo, but rather nicer to use, and can send text to the clipboard.  You can get a lot of information by using SysExp and WinScraper in tandem; what one won't get, the other sometimes will.

[mouser]

Fanix have some very good low-level code, if nothing else Hilitext will be a good way to see how well such an operation can be done and how reliably.

[urlwolf]

thanks rjbull, I don't know how I forgot Hilitext! Great product.
I re-installed it, and surprisingly this time (v. 1.5) hilitext failed systematically on every single application I test it on!

I remember when hilitext was free (you can still get the old version)... it worked in many places (word, even the grumpy opera).
Now, with word 2007 on my computer, no dice. Adobe: nada. Onenote: nitsch.

I have contacted the author to see if he has this functionality available as a library, but that was before I tested it .

Damn, if this is state of the art, it's dissappointing!

SysExporter: Very efficient, it goes down to each window component.
   Freeware
   Gets plenty of stuff, even drop-down boxes!
   It fails getting text from adobe
Word doesn't even show up on the list

Same thing with windowscrapper and showin. Not much text captured here, only window ID, window coordinates,etc. Very much in like with autoIT spy.

Thanks

[yksyks]

How about HandyFind (http://www.handykeys.com)?

[Armando]

Does HandyFind work in any windows? I remember having problem in acrobat with pdf documents, I think.

[rjbull]

thanks rjbull, I don't know how I forgot Hilitext! Great product.
I re-installed it, and surprisingly this time (v. 1.5) hilitext failed systematically on every single application I test it on!

-urlwolf (October 23, 2007, 01:34 PM)

Works for me on most things, including Word 2003 on a brief test (I try not to use Word), but not (again on a brief test) in Acrobat Reader.  PDFs seem problematical anyway.  Kleptomania can't always read them correctly - can't always read Crystal Reports either, usually because the fonts are too small.

I'm having a problem with Hilitext 1.5.1 myself.  It doesn't seem to be honouring its "Highlight a Text in a File" feature.  I e-mailed Fanix support yesterday.

SysExporter: Very efficient, it goes down to each window component.
   Freeware
   Gets plenty of stuff, even drop-down boxes!

But if you've got a lot of windows on screen, and they contain a lot of boxes, it takes a lot of hawk-eyed scrolling to find the right one.  I've thought of suggesting it should present them in an Explorer-style tree and/or have a draggable crosshair like WinScraper and ShoWin do, to home in on your target more quickly.

Same thing with windowscrapper and showin. Not much text captured here, only window ID, window coordinates,etc. Very much in like with autoIT spy.

ShoWin can send those data to the clipboard - I never figured an easy way to do that with AutoIT Spy - and WinScraper will sometimes get data from windows that SysExp won't, so I keep them all around.

But I agree, it isn't a sufficiently happy hunting ground   

[yksyks]

Sorry for mentioning HandyFind. I just installed the latest version 2.0.8 and the only working function is "Donate". It used to work before and seemed to be quite useful. I apologize.

[f0dder]

Sorry for this out-of-topic, but I simply couldn't resist