All AutoHotkey (AHK) coders read: No more upx packing of compiled ahk utils

[mouser]

As of today, I am advocating that we no longer host any compiled ahk utilities that have been "packed" with upx, which is something that autohotkey does by default when it builds exes.

The process of packing the executable with UPX results in a smaller sized file, but causes a continuous an inevitable false virus malware alert sooner or later.

Any application packed with upx by ahk compilation is almost guaranteed to be marked as a virus sooner or later and is going to cause trouble for any site that hosts it, and any users who get scared by it.

The true fault of this lies with the antivirus programs, but until we can get them to stop their bullshit, this is the only thing we have control over.


SO: If you are an ahk coder, go to your autohotkey /Compiler folder and delete the upx.exe executable.  That will stop ahk from packing your executable with UPX, and should solve the problem.


Again i repeat -- if you use AHK, do not upload to this site any compiled ahk program that is packed with upx -- remove the upx.exe from your ahk and recompile please.

[f0dder]

Hrm, is disabling UPX packing enough to not get AHK scripts flagged? I thought that every virus scanner today knows how to unpack UPX (and several other packers) and do the scanning on the unpacked executable.

[mouser]

if that's the case, then things are worse than i thought.

[f0dder]

Even back in the pre-Win9x DOS days, ThunderByte AntiVirus (TBAV, which was the product back then) could decompress exepackers, and it even had a "virtualization" mode for unknown packers¹ - afaik today's antivirus products, at least the better ones, have fast depackers for known exepackers and emulation for unknowns.

I assume the problem with all those false positives is static (or pattern-based) signatures that are simply too short... or heuristic engines that get confused for whatever fscktarded reason.

1: and there was at least one virus that figured out how to break out of the sandboxed mode, in effect causing a virus scan to infect your system

[Stoic Joker]

Even tho I loved UPXs efficiency I quit using it years ago because of the tendency to FP on anything packed with it (Which is the root of why I hate heuristics). I just got tired of being cut off while trying to run a diagnostic on site because the client's AV ate my tool.

Is is "fair" (to UPX) to impose this limitation? No. But it does appear to be necessary. Even if it is only to eliminate it as the culprit...and/or expose a larger problem if it exists.

m2c

[f0dder]

The AV vendors really need to get their sh!t together if merely being packed with a (non-modified) UPX sends their heuristics engines off the tracks... come on, it's easy to identify and unpack.

That said, I'm generally against packing of executable unless there's special reason for it; false positives isn't the only negative aspect of exepacking.

[Stoic Joker]

That said, I'm generally against packing of executable unless there's special reason for it; false positives isn't the only negative aspect of exepacking.

I never really got into the architecture stuff quite that far ... but I can follow it. I will have to (once again) conclude (as I did the first time I read that) that you have a solid case/argument against packers (I did already mention that I'd quite using them above...).

[Nod5]

I completely understand the worries here. All my small programs fall into this category and some have been FP flagged before. I see that Mouser also posted at the AHK forum: http://www.autohotkey.com/forum/topic53129.html . Let's see what folks there reply. As mouser says (in that thread), as long UPX packing is default mode for Autohotkey any site admin that wants to avoid FP deflecting work would have to manually monitor compiled AHK uploads for UPX. Quick idea: maybe autohotkey could be changed so that is uses different colored icons for files compiled with UPX (standard green H) and withoug UPX (same H but blue background, for example)?  That would make spotting the UPX ones easier once you've got the files down into a folder.

[ewemoa]

SO: If you are an ahk coder, go to your autohotkey /Compiler folder and delete the upx.exe executable.

For reference, renaming the upx.exe executable seems to be enough.

In my case I made a duplicate of the Compiler folder with a different name (plus renaming upx.exe) and pointed the custom scripts that build my AHK executables at the Ahk2Exe.exe that lives in this folder.  That seemed to work. 

So the contents of my AutoHotkey folder now look like:

• AutoScriptWriter\
• Compiler\
• CompilerNoUPX\  <-- this is the new folder that contains a renamed upx.exe
• Extras\
• AU3_Spy.exe
• AutoHotkey Website
• AutoHotkey.chm
• AutoHotkey.exe
• license.txt

I still have the option of using upx.exe locally -- though reading f0dder's "Packing, data handling, stuff - revision 2" posted above (thanks f0dder!) I wonder if I'll ever have use for upx.exe for the sorts of things I'm likely to do.

[f0dder]

though reading f0dder's "Packing, data handling, stuff - revision 2" posted above (thanks f0dder!) I wonder if I'll ever have use for upx.exe for the sorts of things I'm likely to do.

Keep in mind that I'm not flat-all-out-against exepacking - it makes sense, for instance, if you distribute small tools as .exe downloads rather than .zip (which isn't necessarily a good idea, but for some stuff it's convenient). Or if your application is designed to occupy very little space (fSekrit comes to mind, since it saves it's executable with every encrypted note you make).

But for most stuff, you really should let the user decide, for the reasons mentioned in my article. Oh, and I see that the article doesn't even mention Windows Terminal Servers - even if your app is single-instance (and you thus don't expect much gain from code/data page sharing), on WTS it could be multiple-instance (across multiple user accounts, of course).

There's also the issue of virus scanners not just being anal about compression, but also scanning the files quite a bit slower than non-packed executables... I recall Jibz (iirc) being annoyed with FileZilla by default being compressed. And I've worked with machines slow enough that you could definitely tell loading speed difference, at least when the machine had antivirus software running.

This is drifting slightly off-topic - I'm pretty good at that

[mouser]

There is a new poll on the UPX form asking whether people approve of the idea of changing AHK compiler behavior to not use UPX by default.
If you have an opinion on this please vote: http://www.autohotkey.com/forum/viewtopic.php?t=53219

[Apathetic_Coding]

This tool written by ladiko over at the AHK forum not only gives you more options when compiling your AHK scripts but also has an option to NOT use UPX.

http://www.autohotkey.com/forum/viewtopic.php?t=22975&highlight=compile

[mouser]

Oh very nice find. 

[jgpaiva]

I just updated GridMove with a non-compressed executable.
Let's hope it get us less problems!

[MilesAhead]

Looks like UPX is a bit smarter about compression of exe/resources than 7-zip. Using the Compile II mentioned with UPX off a trivial AHK script with icon resource compiles to about 486 KB instead of 277 KB.  After compressing both with 7-zip the one with no UPX compresses to a 258 KB download instead of 213 KB. I haven't done any trials with larger apps since I only use AHK for very small scripts.

With the rate these viruses propagate has anyone done any calculations on the possibility of random signature matches?  Or are the permutations so large that it's trivial?  Before Linux had to be able to handle all these Windows file formats, most files didn't even have an extension.  You relied on a program called 'file' iirc that looked at the start of the file and guessed if it was a printer configuration file, a bash script, an executable etc..
I wonder how long it will be until just any random string of bytes at the start of a file matches some virus signature somewhere?

[f0dder]

Before Linux had to be able to handle all these Windows file formats, most files didn't even have an extension.  You relied on a program called 'file' iirc that looked at the start of the file and guessed if it was a printer configuration file, a bash script, an executable etc..

I've always preferred file extensions over having to look at contents (or metadata, as in case of Apple OSes) - for a lot of reasons. Like, being faster and easier to filter by.

[ewemoa]

This tool written by ladiko over at the AHK forum not only gives you more options when compiling your AHK scripts but also has an option to NOT use UPX.

http://www.autohotkey.com/forum/viewtopic.php?t=22975&highlight=compile

FWIW, I've been using this for a few things and it seems pretty decent.  Thanks for pointing this out

[mouser]

a recent post shows that the flagging of everything made with UPX is getting worse and more widespread:
http://www.donationcoder.com/forum/index.php?topic=22582.msg203503#msg203503

STOP USING UPX NOW -- IF YOU HAVE SOFTWARE THAT USES UPX, REBUILD IT WITHOUT UPX

[lanux128]

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files.

[Target]

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files.

can't we just run a virus scan?

[lanux128]

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files.

can't we just run a virus scan?

i hope you're being sarcastic..

[Target]

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files.

can't we just run a virus scan?

i hope you're being sarcastic..

as if...

[lanux128]

we need a coding snack that searches through DC forums and displays links to upx-compiled exe files.

can't we just run a virus scan?

i hope you're being sarcastic..

as if...

ok, i lost you there..

[Target]

nope, still being sarcastic (what can I say, I'm a sarcopath )

[cranioscopical]

I'm a sarcopath)

Please, get off the path and back  to your phagus!