Home | Blog | Software | Reviews and Features | Forum | Help | Donate | About us
topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • December 06, 2016, 12:03:01 PM
  • Proudly celebrating 10 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: GUI automation  (Read 2962 times)

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
GUI automation
« on: September 04, 2011, 05:05:09 AM »
hello

I have tried several GUI automation solutions, including AHK, AutoIt, Ranorex, Sikuli, etc.

The problem was always the same: there is no way to identify many GUI elements/controls.

is there any solution to this problem?

I need to record macros to programs that have GUI elements/controls, that do not have classnn, or other distinguishable characteristics.

thanks

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 2,779
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #1 on: September 04, 2011, 05:45:32 AM »
I need to record macros to programs that have GUI elements/controls, that do not have classnn, or other distinguishable characteristics.
Eventually, most applications have them, but they can hide it in certain situations. When building WinSendKeys I had a bit similar issues, and reverted to doing other-control-relative mouse-clicks, usually it's toolbarbuttons that don't have these classnames/classnn, but the toolbar or it's surrounding windows has, and the toolbar is often in a fixed location.

Can you give examples of applications that hide (most) controls like this? I know Firefox 4.0 and later has issues in this arena, and some apps created using C++Builder 6 have the issues with the toolbars, mentioned above, but I haven't found much other applications with this behavior.

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #2 on: September 04, 2011, 07:28:39 AM »
there are so many controls that are not identifiable, for example in Pop Peeper:
I hover over the Close button, and there is nothing I see that distinguishes it from eg. the Delete button:
111.png

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,220
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: GUI automation
« Reply #3 on: September 04, 2011, 07:49:24 AM »
Have you looked at this:

http://www.mjtnet.com/

Not sure if it addresses what you need.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #4 on: September 04, 2011, 10:45:27 AM »
Have you looked at this:

http://www.mjtnet.com/

Not sure if it addresses what you need.

I just tried it, it looked promising but unfortunately it did not do what I want
I used a macro recorder to record opening of Pop Peeper main window, opening then an email, then clicking Close, and it only recorded the position and movements of the pointer, not any identifiable info of the windows and controls/elements.

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #5 on: September 06, 2011, 02:19:15 PM »
any other solution???

ldarrambide

  • Supporting Member
  • Joined in 2008
  • **
  • default avatar
  • Posts: 15
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #6 on: September 06, 2011, 04:09:56 PM »
The one I use, and saved me a lot of time from daily chores:

Quick Macros

It's the first program I install when reinstalling windows.

Useful from beginners to gurus

http://www.quickmacros.com/

HTH

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #7 on: September 06, 2011, 04:25:13 PM »
The one I use, and saved me a lot of time from daily chores:

Quick Macros

It's the first program I install when reinstalling windows.

Useful from beginners to gurus

http://www.quickmacros.com/

HTH

seems very interesting

can it take screenshots of the elements I click and create computer-vision-driven identification, like Sikuli?

I also found www.mouserobot.com which is also very promising, with an excellent element region identification, but not sure if I can make it identify accurately any element

ldarrambide

  • Supporting Member
  • Joined in 2008
  • **
  • default avatar
  • Posts: 15
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #8 on: September 07, 2011, 01:27:02 AM »
I don't get exactly what means "create computer-vision-driven identification", but
if you mean recognize bitmap on screen and do action on it, yes,
there is a "find image, wait" function that allow image processing.

Go to the forum, very useful, and the author (Gintaras) is
real god of programming and generally answers *any* question
in 24-48h. Responsive and clear.

If not, http://www.macrotoolworks.com/ could have tools
to help you, but i feel less comfortable there....

HTH

bob99

  • Supporting Member
  • Joined in 2008
  • **
  • default avatar
  • Posts: 339
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #9 on: October 12, 2011, 06:24:46 PM »

A while back I did a search on a couple of the different macro recorders and one of the results brought up alternativeTo.net.  It provided a pretty good list of different programs. Link to the list sorted on visitor/user likes: http://alternativeto...k-studio/?sort=likes

I haven't used any of these but one of them may suit your needs or rings a bell with one of the other readers of the post that can provide some input.

Not affiliated with alternativeTo or any of the programs.



kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #10 on: October 13, 2011, 12:24:47 PM »
A while back I did a search on a couple of the different macro recorders and one of the results brought up alternativeTo.net.  It provided a pretty good list of different programs. Link to the list sorted on visitor/user likes: http://alternativeto...k-studio/?sort=likes

I haven't used any of these but one of them may suit your needs or rings a bell with one of the other readers of the post that can provide some input.

Not affiliated with alternativeTo or any of the programs.




thanks for the list, although it is far from complete, since it lacks Ranorex

Jimdoria

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 256
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #11 on: October 13, 2011, 03:54:49 PM »
So Sikuli did not work? Seems from the description like it should wok on any visible screen element. What was the problem?

I'd like to try Sikuli out, but if it doesn't work as advertised, I won't waste my time.
- Jimdoria ~@>@

There are two kinds of people in the world: Those who divide everybody into two kinds of people, and those who don't.

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #12 on: October 13, 2011, 04:09:34 PM »
So Sikuli did not work? Seems from the description like it should wok on any visible screen element. What was the problem?

I'd like to try Sikuli out, but if it doesn't work as advertised, I won't waste my time.
it works very well, but I would like something more accurate

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,473
    • View Profile
    • Donate to Member
Re: GUI automation
« Reply #13 on: October 13, 2011, 04:11:11 PM »
I have an idea

when a specific window control is displayed on screen, specific bytes are in RAM (I suppose)
so why not use that data to identify and manage window controls?