Welcome Guest.   Make a donation to an author on the site July 24, 2014, 10:23:03 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
The N.A.N.Y. Challenge 2010! Download 24 custom programs!
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: All AutoHotkey (AHK) coders read: No more upx packing of compiled ahk utils  (Read 26751 times)
mouser
First Author
Administrator
*****
Posts: 33,177



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: January 08, 2010, 05:48:45 AM »

As of today, I am advocating that we no longer host any compiled ahk utilities that have been "packed" with upx, which is something that autohotkey does by default when it builds exes.

The process of packing the executable with UPX results in a smaller sized file, but causes a continuous an inevitable false virus malware alert sooner or later.

Any application packed with upx by ahk compilation is almost guaranteed to be marked as a virus sooner or later and is going to cause trouble for any site that hosts it, and any users who get scared by it.

The true fault of this lies with the antivirus programs, but until we can get them to stop their bullshit, this is the only thing we have control over.



SO: If you are an ahk coder, go to your autohotkey /Compiler folder and delete the upx.exe executable.  That will stop ahk from packing your executable with UPX, and should solve the problem.



Again i repeat -- if you use AHK, do not upload to this site any compiled ahk program that is packed with upx -- remove the upx.exe from your ahk and recompile please.
« Last Edit: January 08, 2010, 06:44:28 AM by mouser » Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #1 on: January 08, 2010, 06:02:48 AM »

Hrm, is disabling UPX packing enough to not get AHK scripts flagged? I thought that every virus scanner today knows how to unpack UPX (and several other packers) and do the scanning on the unpacked executable.
Logged

- carpe noctem
mouser
First Author
Administrator
*****
Posts: 33,177



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: January 08, 2010, 06:13:10 AM »

if that's the case, then things are worse than i thought.
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: January 08, 2010, 06:19:52 AM »

Even back in the pre-Win9x DOS days, ThunderByte AntiVirus (TBAV, which was the product back then) could decompress exepackers, and it even had a "virtualization" mode for unknown packers1 - afaik today's antivirus products, at least the better ones, have fast depackers for known exepackers and emulation for unknowns.

I assume the problem with all those false positives is static (or pattern-based) signatures that are simply too short... or heuristic engines that get confused for whatever fscktarded reason.

1: and there was at least one virus that figured out how to break out of the sandboxed mode, in effect causing a virus scan to infect your system smiley
« Last Edit: January 08, 2010, 06:55:36 AM by f0dder » Logged

- carpe noctem
Stoic Joker
Honorary Member
**
Posts: 5,097



View Profile WWW Give some DonationCredits to this forum member
« Reply #4 on: January 08, 2010, 06:46:12 AM »

Even tho I loved UPXs efficiency I quit using it years ago because of the tendency to FP on anything packed with it (Which is the root of why I hate heuristics). I just got tired of being cut off while trying to run a diagnostic on site because the client's AV ate my tool.

Is is "fair" (to UPX) to impose this limitation? No. But it does appear to be necessary. Even if it is only to eliminate it as the culprit...and/or expose a larger problem if it exists.

m2c
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: January 08, 2010, 06:54:39 AM »

The AV vendors really need to get their sh!t together if merely being packed with a (non-modified) UPX sends their heuristics engines off the tracks... come on, it's easy to identify and unpack.

That said, I'm generally against packing of executable unless there's special reason for it; false positives isn't the only negative aspect of exepacking.
Logged

- carpe noctem
Stoic Joker
Honorary Member
**
Posts: 5,097



View Profile WWW Give some DonationCredits to this forum member
« Reply #6 on: January 08, 2010, 08:13:42 AM »

That said, I'm generally against packing of executable unless there's special reason for it; false positives isn't the only negative aspect of exepacking.
I never really got into the architecture stuff quite that far ... but I can follow it. I will have to (once again) conclude (as I did the first time I read that) that you have a solid case/argument against packers (I did already mention that I'd quite using them above...).
Logged
Nod5
Supporting Member
**
Posts: 725



View Profile Give some DonationCredits to this forum member
« Reply #7 on: January 08, 2010, 03:21:57 PM »

I completely understand the worries here. All my small programs fall into this category and some have been FP flagged before. I see that Mouser also posted at the AHK forum: http://www.autohotkey.com/forum/topic53129.html . Let's see what folks there reply. As mouser says (in that thread), as long UPX packing is default mode for Autohotkey any site admin that wants to avoid FP deflecting work would have to manually monitor compiled AHK uploads for UPX. Quick idea: maybe autohotkey could be changed so that is uses different colored icons for files compiled with UPX (standard green H) and withoug UPX (same H but blue background, for example)?  That would make spotting the UPX ones easier once you've got the files down into a folder.
« Last Edit: January 08, 2010, 03:38:01 PM by Nod5 » Logged
ewemoa
Honorary Member
**
Posts: 2,387



View Profile Give some DonationCredits to this forum member
« Reply #8 on: January 08, 2010, 05:45:26 PM »

SO: If you are an ahk coder, go to your autohotkey /Compiler folder and delete the upx.exe executable.
For reference, renaming the upx.exe executable seems to be enough.

In my case I made a duplicate of the Compiler folder with a different name (plus renaming upx.exe) and pointed the custom scripts that build my AHK executables at the Ahk2Exe.exe that lives in this folder.  That seemed to work. 

So the contents of my AutoHotkey folder now look like:

  • AutoScriptWriter\
  • Compiler\
  • CompilerNoUPX\  <-- this is the new folder that contains a renamed upx.exe
  • Extras\
  • AU3_Spy.exe
  • AutoHotkey Website
  • AutoHotkey.chm
  • AutoHotkey.exe
  • license.txt

I still have the option of using upx.exe locally -- though reading f0dder's "Packing, data handling, stuff - revision 2" posted above (thanks f0dder!) I wonder if I'll ever have use for upx.exe for the sorts of things I'm likely to do.
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #9 on: January 09, 2010, 04:12:59 AM »

though reading f0dder's "Packing, data handling, stuff - revision 2" posted above (thanks f0dder!) I wonder if I'll ever have use for upx.exe for the sorts of things I'm likely to do.
Keep in mind that I'm not flat-all-out-against exepacking - it makes sense, for instance, if you distribute small tools as .exe downloads rather than .zip (which isn't necessarily a good idea, but for some stuff it's convenient). Or if your application is designed to occupy very little space (fSekrit comes to mind, since it saves it's executable with every encrypted note you make).

But for most stuff, you really should let the user decide, for the reasons mentioned in my article. Oh, and I see that the article doesn't even mention Windows Terminal Servers smiley - even if your app is single-instance (and you thus don't expect much gain from code/data page sharing), on WTS it could be multiple-instance (across multiple user accounts, of course).

There's also the issue of virus scanners not just being anal about compression, but also scanning the files quite a bit slower than non-packed executables... I recall Jibz (iirc) being annoyed with FileZilla by default being compressed. And I've worked with machines slow enough that you could definitely tell loading speed difference, at least when the machine had antivirus software running.

This is drifting slightly off-topic - I'm pretty good at that smiley
« Last Edit: January 09, 2010, 04:16:11 AM by f0dder » Logged

- carpe noctem
mouser
First Author
Administrator
*****
Posts: 33,177



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: January 10, 2010, 12:53:55 PM »

There is a new poll on the UPX form asking whether people approve of the idea of changing AHK compiler behavior to not use UPX by default.
If you have an opinion on this please vote: http://www.autohotkey.com...rum/viewtopic.php?t=53219
Logged
Apathetic_Coding
Honorary Member
**
Posts: 27



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #11 on: January 10, 2010, 01:20:04 PM »

This tool written by ladiko over at the AHK forum not only gives you more options when compiling your AHK scripts but also has an option to NOT use UPX.

http://www.autohotkey.com/forum/viewtopic.php?t=22975&highlight=compile
Logged
mouser
First Author
Administrator
*****
Posts: 33,177



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: January 10, 2010, 01:29:00 PM »

Oh very nice find.  thumbs up
Logged
jgpaiva
Global Moderator
*****
Posts: 4,710



Artificial Idiocy

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #13 on: January 10, 2010, 05:02:58 PM »

I just updated GridMove with a non-compressed executable.
Let's hope it get us less problems!
Logged

MilesAhead
Member
**
Posts: 4,682



View Profile WWW Give some DonationCredits to this forum member
« Reply #14 on: January 13, 2010, 01:12:55 PM »

Looks like UPX is a bit smarter about compression of exe/resources than 7-zip. Using the Compile II mentioned with UPX off a trivial AHK script with icon resource compiles to about 486 KB instead of 277 KB.  After compressing both with 7-zip the one with no UPX compresses to a 258 KB download instead of 213 KB. I haven't done any trials with larger apps since I only use AHK for very small scripts.

With the rate these viruses propagate has anyone done any calculations on the possibility of random signature matches?  Or are the permutations so large that it's trivial?  Before Linux had to be able to handle all these Windows file formats, most files didn't even have an extension.  You relied on a program called 'file' iirc that looked at the start of the file and guessed if it was a printer configuration file, a bash script, an executable etc..
I wonder how long it will be until just any random string of bytes at the start of a file matches some virus signature somewhere?


« Last Edit: January 13, 2010, 01:15:01 PM by MilesAhead » Logged

f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: January 13, 2010, 05:19:13 PM »

Before Linux had to be able to handle all these Windows file formats, most files didn't even have an extension.  You relied on a program called 'file' iirc that looked at the start of the file and guessed if it was a printer configuration file, a bash script, an executable etc..
I've always preferred file extensions over having to look at contents (or metadata, as in case of Apple OSes) - for a lot of reasons. Like, being faster and easier to filter by.
Logged

- carpe noctem
ewemoa
Honorary Member
**
Posts: 2,387



View Profile Give some DonationCredits to this forum member
« Reply #16 on: March 14, 2010, 01:47:58 AM »

This tool written by ladiko over at the AHK forum not only gives you more options when compiling your AHK scripts but also has an option to NOT use UPX.

http://www.autohotkey.com/forum/viewtopic.php?t=22975&highlight=compile

FWIW, I've been using this for a few things and it seems pretty decent.  Thanks for pointing this out smiley
Logged
mouser
First Author
Administrator
*****
Posts: 33,177



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: April 29, 2010, 12:19:24 PM »

a recent post shows that the flagging of everything made with UPX is getting worse and more widespread:
http://www.donationcoder....22582.msg203503#msg203503

STOP USING UPX NOW -- IF YOU HAVE SOFTWARE THAT USES UPX, REBUILD IT WITHOUT UPX
Logged
lanux128
Global Moderator
*****
Posts: 6,087



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: June 07, 2010, 12:28:08 AM »

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. smiley
Logged

Target
Honorary Member
**
Posts: 1,382



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #19 on: June 07, 2010, 01:31:47 AM »

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. smiley

can't we just run a virus scan?
Logged

"Look wise, say nothing, and grunt. Speech was given to conceal thought" - Sir William Osler
lanux128
Global Moderator
*****
Posts: 6,087



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: June 07, 2010, 01:53:48 AM »

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. smiley

can't we just run a virus scan?

i hope you're being sarcastic.. Grin
Logged

Target
Honorary Member
**
Posts: 1,382



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #21 on: June 07, 2010, 02:03:42 AM »

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. smiley

can't we just run a virus scan?

i hope you're being sarcastic.. Grin

as if...

Logged

"Look wise, say nothing, and grunt. Speech was given to conceal thought" - Sir William Osler
lanux128
Global Moderator
*****
Posts: 6,087



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #22 on: June 07, 2010, 02:18:24 AM »

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. smiley

can't we just run a virus scan?

i hope you're being sarcastic.. Grin

as if...

ok, i lost you there.. undecided
Logged

Target
Honorary Member
**
Posts: 1,382



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #23 on: June 07, 2010, 05:03:12 AM »

nope, still being sarcastic (what can I say, I'm a sarcopath embarassed)
Logged

"Look wise, say nothing, and grunt. Speech was given to conceal thought" - Sir William Osler
cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,164



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #24 on: June 07, 2010, 03:35:04 PM »

I'm a sarcopath)
Please, get off the path and back  to your phagus!
Logged

Chris
Pages: [1] 2 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.057s | Server load: 0.08 ]