When programs override the minimize, maximize or close buttons on the frame, there's no standardized way that I've found to predict what action will happen or if it will detect the current state it is in if it didn't cause it internally.
I wrote a "single instance" component where I try to maximize the already running instance before the second instance quits. In the end I had to add an option to turn that attempt off because it could produce weird results like what you mention here. If the program minimized "normally" with a button on the taskbar, usually things worked as expected. But if the program minimized to tray, once you acted on it from outside all bets were off.
About all I can suggest is mess with it by sending it Hide and Show messages, if it has a System Menu, try that and see what happens.
I just messed around with Gigaget download manager which has no option to start in the tray that I know of, but if you click the 'x' on the frame, it will minimize to tray with no task bar button. Great. So I send it a 'hide' msg with NirCmd. Now it won't open normally. The only solution was to run another program that made Gigaget the active window and sent it an Alt-F4 since it handled that as minimize to tray also.
Every program is going to be different. Unless you run one of those monitor things that can figure it out, then it's pretty much trial and error.
At least afaik. I haven't done one of those tray minimizer hider apps. Someone that has will likely have more insight.
