Home | Blog | Software | Reviews and Features | Forum | Help | Donate | About us
topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • September 05, 2015, 02:47:36 AM
  • Proudly celebrating 10 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: All AutoHotkey (AHK) coders read: No more upx packing of compiled ahk utils  (Read 34325 times)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 35,013
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Donate to Member
As of today, I am advocating that we no longer host any compiled ahk utilities that have been "packed" with upx, which is something that autohotkey does by default when it builds exes.

The process of packing the executable with UPX results in a smaller sized file, but causes a continuous an inevitable false virus malware alert sooner or later.

Any application packed with upx by ahk compilation is almost guaranteed to be marked as a virus sooner or later and is going to cause trouble for any site that hosts it, and any users who get scared by it.

The true fault of this lies with the antivirus programs, but until we can get them to stop their bullshit, this is the only thing we have control over.



SO: If you are an ahk coder, go to your autohotkey /Compiler folder and delete the upx.exe executable.  That will stop ahk from packing your executable with UPX, and should solve the problem.



Again i repeat -- if you use AHK, do not upload to this site any compiled ahk program that is packed with upx -- remove the upx.exe from your ahk and recompile please.
« Last Edit: January 08, 2010, 06:44:28 AM by mouser »

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 8,858
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Donate to Member
Hrm, is disabling UPX packing enough to not get AHK scripts flagged? I thought that every virus scanner today knows how to unpack UPX (and several other packers) and do the scanning on the unpacked executable.
- carpe noctem

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 35,013
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Donate to Member
if that's the case, then things are worse than i thought.

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 8,858
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Donate to Member
Even back in the pre-Win9x DOS days, ThunderByte AntiVirus (TBAV, which was the product back then) could decompress exepackers, and it even had a "virtualization" mode for unknown packers1 - afaik today's antivirus products, at least the better ones, have fast depackers for known exepackers and emulation for unknowns.

I assume the problem with all those false positives is static (or pattern-based) signatures that are simply too short... or heuristic engines that get confused for whatever fscktarded reason.

1: and there was at least one virus that figured out how to break out of the sandboxed mode, in effect causing a virus scan to infect your system :)
- carpe noctem
« Last Edit: January 08, 2010, 06:55:36 AM by f0dder »

Stoic Joker

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 5,883
    • View Profile
    • www.StoicJoker.com
    • Donate to Member
Even tho I loved UPXs efficiency I quit using it years ago because of the tendency to FP on anything packed with it (Which is the root of why I hate heuristics). I just got tired of being cut off while trying to run a diagnostic on site because the client's AV ate my tool.

Is is "fair" (to UPX) to impose this limitation? No. But it does appear to be necessary. Even if it is only to eliminate it as the culprit...and/or expose a larger problem if it exists.

m2c

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 8,858
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Donate to Member
The AV vendors really need to get their sh!t together if merely being packed with a (non-modified) UPX sends their heuristics engines off the tracks... come on, it's easy to identify and unpack.

That said, I'm generally against packing of executable unless there's special reason for it; false positives isn't the only negative aspect of exepacking.
- carpe noctem

Stoic Joker

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 5,883
    • View Profile
    • www.StoicJoker.com
    • Donate to Member
That said, I'm generally against packing of executable unless there's special reason for it; false positives isn't the only negative aspect of exepacking.
I never really got into the architecture stuff quite that far ... but I can follow it. I will have to (once again) conclude (as I did the first time I read that) that you have a solid case/argument against packers (I did already mention that I'd quite using them above...).

Nod5

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 814
    • View Profile
    • Donate to Member
I completely understand the worries here. All my small programs fall into this category and some have been FP flagged before. I see that Mouser also posted at the AHK forum: http://www.autohotke...orum/topic53129.html . Let's see what folks there reply. As mouser says (in that thread), as long UPX packing is default mode for Autohotkey any site admin that wants to avoid FP deflecting work would have to manually monitor compiled AHK uploads for UPX. Quick idea: maybe autohotkey could be changed so that is uses different colored icons for files compiled with UPX (standard green H) and withoug UPX (same H but blue background, for example)?  That would make spotting the UPX ones easier once you've got the files down into a folder.
« Last Edit: January 08, 2010, 03:38:01 PM by Nod5 »

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • default avatar
  • Posts: 2,718
    • View Profile
    • Donate to Member
SO: If you are an ahk coder, go to your autohotkey /Compiler folder and delete the upx.exe executable.
For reference, renaming the upx.exe executable seems to be enough.

In my case I made a duplicate of the Compiler folder with a different name (plus renaming upx.exe) and pointed the custom scripts that build my AHK executables at the Ahk2Exe.exe that lives in this folder.  That seemed to work. 

So the contents of my AutoHotkey folder now look like:

  • AutoScriptWriter\
  • Compiler\
  • CompilerNoUPX\  <-- this is the new folder that contains a renamed upx.exe
  • Extras\
  • AU3_Spy.exe
  • AutoHotkey Website
  • AutoHotkey.chm
  • AutoHotkey.exe
  • license.txt

I still have the option of using upx.exe locally -- though reading f0dder's "Packing, data handling, stuff - revision 2" posted above (thanks f0dder!) I wonder if I'll ever have use for upx.exe for the sorts of things I'm likely to do.

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 8,858
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Donate to Member
though reading f0dder's "Packing, data handling, stuff - revision 2" posted above (thanks f0dder!) I wonder if I'll ever have use for upx.exe for the sorts of things I'm likely to do.
Keep in mind that I'm not flat-all-out-against exepacking - it makes sense, for instance, if you distribute small tools as .exe downloads rather than .zip (which isn't necessarily a good idea, but for some stuff it's convenient). Or if your application is designed to occupy very little space (fSekrit comes to mind, since it saves it's executable with every encrypted note you make).

But for most stuff, you really should let the user decide, for the reasons mentioned in my article. Oh, and I see that the article doesn't even mention Windows Terminal Servers :) - even if your app is single-instance (and you thus don't expect much gain from code/data page sharing), on WTS it could be multiple-instance (across multiple user accounts, of course).

There's also the issue of virus scanners not just being anal about compression, but also scanning the files quite a bit slower than non-packed executables... I recall Jibz (iirc) being annoyed with FileZilla by default being compressed. And I've worked with machines slow enough that you could definitely tell loading speed difference, at least when the machine had antivirus software running.

This is drifting slightly off-topic - I'm pretty good at that :)
- carpe noctem
« Last Edit: January 09, 2010, 04:16:11 AM by f0dder »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 35,013
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Donate to Member
There is a new poll on the UPX form asking whether people approve of the idea of changing AHK compiler behavior to not use UPX by default.
If you have an opinion on this please vote: http://www.autohotke...iewtopic.php?t=53219

Apathetic_Coding

  • Honorary Member
  • Joined in 2010
  • **
  • Posts: 27
    • View Profile
    • Apathy Softworks
    • Donate to Member
This tool written by ladiko over at the AHK forum not only gives you more options when compiling your AHK scripts but also has an option to NOT use UPX.

http://www.autohotkey.com/forum/viewtopic.php?t=22975&highlight=compile

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 35,013
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Donate to Member
Oh very nice find.  :up:

jgpaiva

  • Global Moderator
  • Joined in 2006
  • *****
  • Posts: 4,727
    • View Profile
    • Read more about this member.
    • Donate to Member
I just updated GridMove with a non-compressed executable.
Let's hope it get us less problems!

MilesAhead

  • Supporting Member
  • Joined in 2009
  • **
  • Posts: 6,202
    • View Profile
    • Miles Ahead Software
    • Donate to Member
Looks like UPX is a bit smarter about compression of exe/resources than 7-zip. Using the Compile II mentioned with UPX off a trivial AHK script with icon resource compiles to about 486 KB instead of 277 KB.  After compressing both with 7-zip the one with no UPX compresses to a 258 KB download instead of 213 KB. I haven't done any trials with larger apps since I only use AHK for very small scripts.

With the rate these viruses propagate has anyone done any calculations on the possibility of random signature matches?  Or are the permutations so large that it's trivial?  Before Linux had to be able to handle all these Windows file formats, most files didn't even have an extension.  You relied on a program called 'file' iirc that looked at the start of the file and guessed if it was a printer configuration file, a bash script, an executable etc..
I wonder how long it will be until just any random string of bytes at the start of a file matches some virus signature somewhere?


« Last Edit: January 13, 2010, 01:15:01 PM by MilesAhead »

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 8,858
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Donate to Member
Before Linux had to be able to handle all these Windows file formats, most files didn't even have an extension.  You relied on a program called 'file' iirc that looked at the start of the file and guessed if it was a printer configuration file, a bash script, an executable etc..
I've always preferred file extensions over having to look at contents (or metadata, as in case of Apple OSes) - for a lot of reasons. Like, being faster and easier to filter by.
- carpe noctem

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • default avatar
  • Posts: 2,718
    • View Profile
    • Donate to Member
This tool written by ladiko over at the AHK forum not only gives you more options when compiling your AHK scripts but also has an option to NOT use UPX.

http://www.autohotkey.com/forum/viewtopic.php?t=22975&highlight=compile

FWIW, I've been using this for a few things and it seems pretty decent.  Thanks for pointing this out :)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 35,013
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Donate to Member
a recent post shows that the flagging of everything made with UPX is getting worse and more widespread:
http://www.donationc....msg203503#msg203503

STOP USING UPX NOW -- IF YOU HAVE SOFTWARE THAT USES UPX, REBUILD IT WITHOUT UPX

lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,219
    • View Profile
    • Coding Snacks by Lanux128
    • Donate to Member
we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. :)

Target

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 1,501
    • View Profile
    • Donate to Member
we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. :)

can't we just run a virus scan?

lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,219
    • View Profile
    • Coding Snacks by Lanux128
    • Donate to Member
we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. :)

can't we just run a virus scan?

i hope you're being sarcastic.. ;D

Target

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 1,501
    • View Profile
    • Donate to Member
we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. :)

can't we just run a virus scan?

i hope you're being sarcastic.. ;D

as if...


lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,219
    • View Profile
    • Coding Snacks by Lanux128
    • Donate to Member
we need a coding snack that searches through DC forums and displays links to upx-compiled exe files. :)

can't we just run a virus scan?

i hope you're being sarcastic.. ;D

as if...

ok, i lost you there.. :-\

Target

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 1,501
    • View Profile
    • Donate to Member
nope, still being sarcastic (what can I say, I'm a sarcopath :-[)

cranioscopical

  • Friend of the Site
  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 4,303
    • View Profile
    • Read more about this member.
    • Donate to Member
I'm a sarcopath)
Please, get off the path and back  to your phagus!