Main Area and Open Discussion > General Software Discussion

hard: find a word on a window, highlight it

(1/3) > >>

urlwolf:
This might be a hard problem: locate a word on the screen and highlight it. In ANY type of window (adobe acrobat, a browser, MS word...).

This problem has three parts
1 - getting a reliable text dump
A 'hacky' approach would be to click on the area that displays text, send ctrl a + ctrl c, put that into the clipboard and the operate from there. This is application-general. Nice. A (maybe) cleaner approach would be to write rules for each application to save a text dump (e.g., sending the kb shortcuts for 'save as', 'export' etc depending on each application. It'd generate spaghetti code :), not very elegant :).

I can see that AutoIT windows spy drops two interesting pieces of information under:
>>>>>>>>>>>( Visible Window Text )<<<<<<<<<<<

>>>>>>>>>>>( Hidden Window Text )<<<<<<<<<<<
However this is most of the time uninteresting information such as the name of the controls, buttons, etc, nowhere near the full text dump that I'm thinking about.

A similar approach -anything out there that does like autoIT spy but grabs all text in a window?- would be an elegant solution...
2 - finding the word in the text dump
Here one may need some interesting algorithms for cases where the word appears more than once. To simplify, pick the first occurence.

3- locate the word in the native application! (hard!)
The only application-general way of doing this to do an OCR of a screen-capture of the text area, and then use some proprietary (all decent OCR libraries are proprietary :() function call to find the coordinates of that word. then draw a circle on top to highlight it...

If this was application-specific (e.g., firefox only, maybe one can highlight the word using UML/javascript (the language for FF extensions) to highlight the word.

Am I overcomplicating things? Do you have any tricks/hacks to solve this problem?

Thanks!
(PS: I have posted this in the ahk forum, as I'm using ahk to quickly get a mockup working...)

f0dder:
It's not going to be an easy task, for the exact same reason that doing "scrolling screen capture" is so damn hard - various applications handle things differently, even when there's no good reason to do so. THEORETICALLY you should be able to simply send a WM_GETTEXT window message to toplevel windows, but that won't work for formatting etc... and even if this would work, getting the screen coords of the text is another bother.

mouser:
Yeah, doing this reliably for different kinds of applications may be very difficult.  You might be able to do well on a few specific applications though, and have moderate success in general, but it's going to be a bit painful.

ps. This should probably be in developer's corner.. I'll move it there in a bit.

rjbull:
I'm probably missing something, but FWIW:

If you just want to highlight a word, try Fanix Software's Hilitext.

If you want to extract the highlit text, Structurise's Kleptomania.  Freeware solutions that get some of the information include Nir Sofer's SysExp, Peter Boulton's WinScraper with alternate direct download link http://www.dataper.demon.co.uk/winscraper/WinScraperSetup.exe, and Robin Keir's ShoWin, now owned by Foundstone (link from Robin Keir's page).  Of these, WinScraper and ShoWin both behave rather like AutoIt's WindowSpy or PowerPro's WindowInfo, but rather nicer to use, and can send text to the clipboard.  You can get a lot of information by using SysExp and WinScraper in tandem; what one won't get, the other sometimes will.

mouser:
Fanix have some very good low-level code, if nothing else Hilitext will be a good way to see how well such an operation can be done and how reliably.

Navigation

[0] Message Index

[#] Next page