Author Topic: Which archive software are we using in 2010 (and why)? (Read 40026 times)

wr975 · « **Reply #50 on:** August 13, 2010, 09:04 AM »

I believe you are right about the dump that I receive in RAR format is not packed using the most extreme settings RAR has to offer.
But still, I expect 7-zip to be at least 30% to 50% more efficient than anything winRAR can bring to the table. Regarding my dumpfiles that is.
-Shades (August 12, 2010, 05:52 PM)

At the cost of time.

I tried packing 2 GB of Access databases. 7z (Ultra) was about 26% smaller, but needed 20 minutes, while RAR (best) needed 6 minutes.

26% more effective, but 3x slower.

I'm not sure if "smallest archive size" is the only way to find the "best archiving utility". I think RAR's compression time/size ratio is quite OK. I also like its ability to detected CRC errors in (split) archives, report missing archives, repair defective files using recovery records, keep broken files, extract from split archives even without the full set,... and so on.

Just guessing, but perhaps your 200 GB file is packed with "super fast" because it would need longer to compress / decompress a strong compressed file (7z "ultra" or RAR "best"), than downloading the extra 11 GB (because of weak compression).

CWuestefeld · « **Reply #51 on:** August 13, 2010, 09:24 AM »

I'm not sure if "smallest archive size" is the only way to find the "best archiving utility". I think RAR's compression time/size ratio is quite OK.
-wr975 (August 13, 2010, 09:04 AM)

It certainly is ... for applications where its compression/speed tradeoff is appropriate. Every application has a unique requirement.

I recently had to change from ZIP to 7Z for an application here at work. The archive file size exceeded 2GB, and the limitations of some of the software involved (including our embedded ZIP archive code, if not the ZIP format itself) forced us to hold the file size below that limit. RAR was out of the question, since the compression code requires a commercial license. We had little choice but to pick 7Z in this case.

On the other hand, we've done a fair amount of experimentation on compression of data to be transmitted through web services. Again using the same embedded ZIP compressor, we found that maximal compression only achieved a marginal improvement over high compression (for the type of data we were compressing, at least), but required many times the computation resources. It seemed like the best tradeoff for the scalability of our web services was to go with just high compression, and let the CPUs and RAM have a lighter load.

Shades · « **Reply #52 on:** August 13, 2010, 10:06 PM »

I believe you are right about the dump that I receive in RAR format is not packed using the most extreme settings RAR has to offer.
But still, I expect 7-zip to be at least 30% to 50% more efficient than anything winRAR can bring to the table. Regarding my dumpfiles that is.
-Shades (August 12, 2010, 05:52 PM)

At the cost of time.

I tried packing 2 GB of Access databases. 7z (Ultra) was about 26% smaller, but needed 20 minutes, while RAR (best) needed 6 minutes.

26% more effective, but 3x slower.

I'm not sure if "smallest archive size" is the only way to find the "best archiving utility". I think RAR's compression time/size ratio is quite OK. I also like its ability to detected CRC errors in (split) archives, report missing archives, repair defective files using recovery records, keep broken files, extract from split archives even without the full set,... and so on.

Just guessing, but perhaps your 200 GB file is packed with "super fast" because it would need longer to compress / decompress a strong compressed file (7z "ultra" or RAR "best"), than downloading the extra 11 GB (because of weak compression).
-wr975 (August 13, 2010, 09:04 AM)

Your point is very true...with a high capacity connection.

However, I have to pull those files through a 512KBit connection which is costing about 120 USD/month. You can get cheaper, higher rated connections over here, but those lines here are really 'overbooked' and unreliable. In my situation I simply lose too much time downloading. Besides that, my download is immediately ready to be stored on the least amount of DVD's.

electronixtar · « **Reply #53 on:** August 14, 2010, 01:18 AM »

There is a Chinese WinRAR knockoff for 7z:

http://www.coolrar.com/

source code (will be BSD in 2.0):

http://www.coolrar.c...load/CoolRAR1.0.0.7z

f0dder · « **Reply #54 on:** August 15, 2010, 04:15 PM »

Shades: sending only a diff of the dumps isn't an option?

Shades · « **Reply #55 on:** August 15, 2010, 06:29 PM »

No, not really. (over-)Eager database admins change always so much that a complete dump is necessary.

if it was always the same database then I agree with your point...but unfortunately that isn't the case.

Deozaan · « **Reply #56 on:** August 16, 2010, 12:05 AM »

(creating different archives per file or folder, and deleting the source files after compression are invaluable features)
-Lashiec (August 10, 2010, 06:39 PM)

Thanks for reminding me of the only two features I miss from WinRAR.

However, a bit of investigation led me to an AHK script/utility called 7-Zip Each File. It's a lot easier than trying to figure out how to do it by command-line. The only thing it seems to be missing is the option to delete the source file after compressing.

tslim · « **Reply #57 on:** August 17, 2010, 03:23 PM »

Your point is very true...with a high capacity connection.

However, I have to pull those files through a 512KBit connection which is costing about 120 USD/month. You can get cheaper, higher rated connections over here, but those lines here are really 'overbooked' and unreliable. In my situation I simply lose too much time downloading. Besides that, my download is immediately ready to be stored on the least amount of DVD's.
-Shades (August 13, 2010, 10:06 PM)

I find your way of measuring efficiency odd...
1. It takes you longer to produce the smallest size zip but requires less download time.
or
2. It takes you longer to download a moderate size zip but saves you some compression time.

Either way, you are simply swithcing the "wait" between the 2 processes.

If the zip is prepared by someone else, then he/she always produces a bigger zip to save his/her time, that is a matter of his/her choice and should not be taken as a factor in determining WinRar's efficiency.

Shades · « **Reply #58 on:** August 17, 2010, 07:11 PM »

Your point is very true...with a high capacity connection.

However, I have to pull those files through a 512KBit connection which is costing about 120 USD/month. You can get cheaper, higher rated connections over here, but those lines here are really 'overbooked' and unreliable. In my situation I simply lose too much time downloading. Besides that, my download is immediately ready to be stored on the least amount of DVD's.
-Shades (August 13, 2010, 10:06 PM)

I find your way of measuring efficiency odd...
1. It takes you longer to produce the smallest size zip but requires less download time.
or
2. It takes you longer to download a moderate size zip but saves you some compression time.

Either way, you are simply swithcing the "wait" between the 2 processes.

If the zip is prepared by someone else, then he/she always produces a bigger zip to save his/her time, that is a matter of his/her choice and should not be taken as a factor in determining WinRar's efficiency.

-tslim (August 17, 2010, 03:23 PM)

Granted, you have a point there...if you are using a system (or work-flow) where the bottleneck is almost the same size than the rest of the system.

However you do not seem to grok that the bottleneck in my complete work-flow is the internet connection...let me clarify with an example. Imagine a bucket of water, an empty bucket similar to the first one and a straw to get the water from one bucket to the other.

I use a VPN connection to let a quad core PC create the archive in 2 to 2,5 hours the download will take an x amount of time. Downloading the moderately compressed archive easily takes x + 6 to 8 hours more (if I am lucky, internet here in Paraguay is really not as stable, as in the northern hemisphere of the globe...even with business subscriptions).

Over here I use an i7 PC that takes 30 to 40 minutes to unpack the highly compressed archive and I am already way into the importing process when the moderately compressed would be in. As I am paid by the hour, my benefactor does not complain at all.

Besides that, with all the computing power that is available in modern PC's, do you really think it should be wasted by playing the latest games or editing/encoding your home video alone?

Tuxman · « **Reply #59 on:** August 19, 2010, 06:21 PM »

I switched from PowerArchiver to WinRAR when it became payware until I found 7-Zip some years ago.
Sticking with .7z (LZMA2 is fine) for most purposes, rather exotic file types are handled by IZArc (yes, I still do work with .ace archives every now and then).

Although I'm quite sure that PAQ8 is still the strongest compression algorithm, it lacks speed. Would probably prefer it to .7z else.

fenixproductions · « **Reply #60 on:** August 20, 2010, 08:11 AM »

WinRAR for me because of:
- the best speed/size ratio,
- nice command line support,
- file names encryption (SQX had that but died).

Deozaan · « **Reply #61 on:** August 20, 2010, 04:16 PM »

WinRAR for me because of:
- the best speed/size ratio,
- nice command line support,
- file names encryption (SQX had that but died).
-fenixproductions (August 20, 2010, 08:11 AM)

I don't have (much) experience with either of these, but 7-Zip also has command-line support (not sure how good it is) and file name encryption.

fenixproductions · « **Reply #62 on:** August 20, 2010, 07:30 PM »

I don't have (much) experience with either of these, but 7-Zip also has command-line support (not sure how good it is) and file name encryption.
-Deozaan (August 20, 2010, 04:16 PM)

Now I feel ashamed because for password protection I could swear the opposite. When was it added? Has "error recovery" feature been added too? Either way speed issue remains.

It seems I am too much behind

Deozaan · « **Reply #63 on:** August 21, 2010, 06:48 PM »

I don't have (much) experience with either of these, but 7-Zip also has command-line support (not sure how good it is) and file name encryption.
-Deozaan (August 20, 2010, 04:16 PM)
Now I feel ashamed because for password protection I could swear the opposite. When was it added? Has "error recovery" feature been added too? Either way speed issue remains.
-fenixproductions (August 20, 2010, 07:30 PM)

I don't know when those options were added and I don't know if it has any "error recovery" features. I know it can check to make sure archives are valid, but I suspect that's not what you mean.

tslim · « **Reply #64 on:** August 26, 2010, 08:19 AM »

However you do not seem to grok that the bottleneck in my complete work-flow is the internet connection...
-Shades (August 13, 2010, 10:06 PM)

I understand internet connection speed is the bottleneck and I also understand in your case, the best archiver will be the one which compresses the most.

However, no archive utility relies on internet connection to do its job, so a measure which is based on internet connection speed can not be used to judge how good an archive program is.

CWuestefeld · « **Reply #65 on:** August 26, 2010, 08:26 AM »

a measure which is based on internet connection speed can not be used to judge how good an achieve[sic] program is.
-tslim (August 26, 2010, 08:19 AM)

This is false. You seem to believe that there is a single best program for everyone, that they can be judged in absolute terms.

Really, the only valid measure is how well a given program meets its user's needs. Since each of us are trying to accomplish different things in different ways, it's likely that there will be a range of programs that excel, with some of us choosing different ones as the optimum.

Shades has explained quite clearly (at the risk of belaboring a sub-thread that I think is already a dead horse) that total time to move a chunk of data is his greatest criteria, so obviously the best for him is the program that can compress the greatest. On the other hand, when I'm compressing I'm generally doing it as a backup; with this cumbersome task, I'm much happier when the time of the compression and decompression is optimal, even if it costs me a bit more space. So Shades and I have opposite "best archiver" conclusions, but neither of us is wrong.

And by the way, please take some time to spell-check your posts. Your repeated substitution of "achieve" instead of "archive" took me several reads to figure out.

iphigenie · « **Reply #66 on:** August 28, 2010, 02:29 AM »

I have almost never given much thought to compression - use what comes with the OS (ususally providing a combo of bz2, gz and zip), or little old peazip. On the servers I still tend to stick with gz - files are bigger, but the time and CPU load compressing large log files with the other formats is too disruptive

Although watching people with different priorities in the time vs cpu use vs size debate trying to argue the best product out (nicely) is rather entertaining

Curt · « **Reply #67 on:** September 08, 2010, 10:10 AM »

7-Zip 9.16 Beta: http://majorgeeks.com/7-Zip_d4603.html

tslim · « **Reply #68 on:** September 13, 2010, 12:11 PM »

a measure which is based on internet connection speed can not be used to judge how good an achieve[sic] program is.
-tslim (August 26, 2010, 08:19 AM)

This is false. You seem to believe that there is a single best program for everyone, that they can be judged in absolute terms.
-CWuestefeld (August 26, 2010, 08:26 AM)

No! My statement is not FALSE. It has nothing to do with "single best program for everyone"
I just want to point out that, if there is a formula which measure the power of archive program, then "internet connection speed" can never be a factor of that formula.

In a layman concept, you can of course relate anything as a mean to measure the usefulness of archive program, but any conclusion you made will become subjective and (no offense, please) is a crap. Just like I could say, because my keyboard is crack and therefore very difficult for me to type/input into MS Excel, I therefore find MS Excel is not good.
Would you agree with me that "keyboard" is a solid factor to measure how good MS Excel is?

dhuser · « **Reply #69 on:** September 13, 2010, 08:26 PM »

I use WinRAR for its speed and ease of use, but I also have the PowerArchiver Free version installed as well - They recently released a free version at http://www.powerarchiver.com/blog/2010/07/01/powerarchiver-free-happy-4th-of-july/ . On a less powerful system in my house, I have 7-Zip installed.

Tuxman · « **Reply #70 on:** September 13, 2010, 08:59 PM »

Damn, I missed the PA Free. Although I had tested the latest "Full" version a few weeks ago and wasn't too impressed ...

def · « **Reply #71 on:** September 18, 2010, 01:06 PM »

I notice that nobody uses WinRK. What do you think? I heard the stability should have improved a lot compared to previous versions. Anyone tried the demo of the current version yet? On the other hand, I'm not sure WinRK is still actively developed - the newest version ist two years old now...

I currently use StuffIt on WinXP currently, but only because I found the StuffIt CD before I downloaded 7-Zip. I guess if I encounter a 7z archive I will replace StuffIt with 7-Zip - always liked the lean 7-Zip better than this bloated file manager, backup scheduler, e-mail, FTP... and *cough* compression package called StuffIt...

Curt · « **Reply #72 on:** October 06, 2010, 02:40 AM »

7-Zip 9.16 Beta: http://majorgeeks.com/7-Zip_d4603.html
-Curt (September 08, 2010, 10:10 AM)

7-Zip 9.17 Beta: http://downloads.pca...418-7-zip-917-32-bit

Giampy · « **Reply #73 on:** October 06, 2010, 08:51 AM »

What do you think about ARJ?

MrCrispy · « **Reply #74 on:** October 06, 2010, 04:09 PM »

Anyone have a Coolrar english version? Would like to try it out.