Welcome Guest.   Make a donation to an author on the site September 01, 2014, 02:30:56 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Your Support Funds this Site: View the Supporter Yearbook.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: Prev 1 2 [3] 4 5 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Compressing files into multiple .zip archives with each .zip being independent  (Read 48523 times)
testgames
Participant
*
Posts: 10

View Profile Give some DonationCredits to this forum member
« Reply #50 on: December 10, 2007, 04:56:04 AM »

Hi,

1. we are talking about around a few hundreds of sub-directories, which contain many small .dat files.
2. There are about 11 levels. every level contains around 4-5 levels so the hole tree is about 400,000 different directories (an encoded game). The hole tree is about 7 GB.
3. I don't really understand what do you mean by output mode, but I guess standard.

Logged
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #51 on: December 10, 2007, 05:34:15 AM »

1. we are talking about around a few hundreds of sub-directories, which contain many small .dat files.
2. There are about 11 levels. every level contains around 4-5 levels so the hole tree is about 400,000 different directories (an encoded game). The hole tree is about 7 GB.

Since it's such a huge directory that you're working on, is there any particular reason you want to use SpInZip over any of the much more complete and capable compression apps out there (honest question...I'm not trying to be a smartass)?

3. I don't really understand what do you mean by output mode, but I guess standard.

Sorry, by output mode, I mean which of the three output modes that SpInZip offers were you attempting to use (and the settings you were attempting to use with that output mode)?  If the aforementioned 2GB zip limit is a problem, I can easily add support for .7z files.  However, you will need 7-zip (free @ http://www.7-zip.org) to unpack the resultant files.
Logged

testgames
Participant
*
Posts: 10

View Profile Give some DonationCredits to this forum member
« Reply #52 on: December 12, 2007, 09:54:24 AM »

here is the story:
1. My customer has an old version of Winzip. He can not unzip files more than 2 GB size.
2. I am sending to this customers many games (hugh sized) which are more that 2 GB each.
3. what we did until now is to manually close zip files up to 2 GB and send them seperatly (so he can unzip them one by one). for example: a 7 GB game = 4 separate zip files.
4. He does not have tar or rar or any other unzip mechanizm nor he will not upgrade his winzip version to support more than 2 GB files.
5. I need the zip files to be stand alone - so I can open them by themsleves and not X.zip.001, X.zip.002 which comines together are a part of a bigger zip.
got it ?
Logged
testgames
Participant
*
Posts: 10

View Profile Give some DonationCredits to this forum member
« Reply #53 on: December 13, 2007, 06:52:19 AM »

1. I used the 1st option for Output mode: "By Approzimate Zip File Size" = 2048 MB
2. I don't want to use the 7-zip to unpack the file (the customer will not install it at his network) so I need it to be open only by unzip.

Logged
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #54 on: December 17, 2007, 07:53:38 AM »

4. He does not have tar or rar or any other unzip mechanizm nor he will not upgrade his winzip version to support more than 2 GB files.
5. I need the zip files to be stand alone - so I can open them by themsleves and not X.zip.001, X.zip.002 which comines together are a part of a bigger zip.

Since he won't upgrade, have you tried making a self-extracting spanned zip file that you could send him?

1. I used the 1st option for Output mode: "By Approzimate Zip File Size" = 2048 MB
2. I don't want to use the 7-zip to unpack the file (the customer will not install it at his network) so I need it to be open only by unzip.

The closest directory I have to yours would be my mail server folder.  It currently has 581,395 files in it comprising 12.1GB of space.  I'll try testing against it and see what happens.
Logged

testgames
Participant
*
Posts: 10

View Profile Give some DonationCredits to this forum member
« Reply #55 on: December 18, 2007, 06:02:05 AM »

I can make self extracting spanned zip files (at 2 GB size) but the problem is I have around 400 of these to make and it will take me too much time, therefore I've been looking for a tool to make it.

Testing it on the mail server can be a great test - thanks a lot, looking forward for the results.

Logged
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #56 on: December 18, 2007, 08:06:35 AM »

I can make self extracting spanned zip files (at 2 GB size) but the problem is I have around 400 of these to make and it will take me too much time, therefore I've been looking for a tool to make it.

Gotcha.

Testing it on the mail server can be a great test - thanks a lot, looking forward for the results.

I've made some changes and they look promising.  I'm currently running a test on my mail server directory using similar settings to what you were trying (Option 1, 2000MB).  It seems to be working fine.  The reason I'm not using 2048MB is that, as I mentioned in a previous post, I am not doing any sort of magical algorithm that attempts to make perfects-sized zip files.   I simply recurse through the input folder and keep a running total of the file sizes.  As soon as that total goes OVER your specified size, I zip those files.  Since you're attempting to push the upper limit of a normal zip file, I'd suggest you use something lower than 2048MB since you run the risk of exceeding that amount.  That why I put "approximate" on the app's interface.  Does that make sense?

In addition, I should be able to support zips greater than 2GB as well.  I'll test that next.
Logged

skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #57 on: December 18, 2007, 10:46:47 AM »

Test #1:

[copy or print]
skwire@hankoku /cygdrive/c/4
$ ls -al e*
-rwx------+ 1 skwire None 2037532760 Dec 18 07:51 email-1.zip
-rwx------+ 1 skwire None 2019501177 Dec 18 08:01 email-2.zip
-rwx------+ 1 skwire None 2010861617 Dec 18 08:07 email-3.zip
-rwx------+ 1 skwire None 2007484481 Dec 18 08:13 email-4.zip
-rwx------+ 1 skwire None 2003196991 Dec 18 08:19 email-5.zip
-rwx------+ 1 skwire None 2049079438 Dec 18 08:40 email-6.zip
-rwx------+ 1 skwire None 1058473649 Dec 18 08:48 email-7.zip

Notes:

1) 12GB+ directory with over half a million files in it.
2) Output mode #1, 2000 as the size setting.
3) It took about an hour for all seven zips to be created.  This was using zero (0) for the compression setting (quickest mode).


Test #2:

[copy or print]
skwire@hankoku /cygdrive/c/4
$ ls -al i*
-rwx------+ 1 skwire None 2956007035 Dec 18 09:12 images-1.zip

Notes:

1) Source folder has 3593 files in it comprising 2.75 GB of space.
2) Output mode #1, 3000 as the size setting.
3) Generated file is a 2.75GB zip.
 
Logged

skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #58 on: December 18, 2007, 04:26:29 PM »

Here's the latest: http://www.autohotkey.net...pinzip-setup-v0.0.6.0.exe

Give it a try and let me know if it works any better this time around.  Also, would anybody be interested in encryption or password protection of the generated zip files?
Logged

testgames
Participant
*
Posts: 10

View Profile Give some DonationCredits to this forum member
« Reply #59 on: December 19, 2007, 03:27:46 AM »

wow !!! it works !!!

I've tried it and I got 3 zip files from a 5 GB directory tree.
Thanks a lot, for the quick responses, the clear answers and the willingness to help.
Cheers !
Logged
testgames
Participant
*
Posts: 10

View Profile Give some DonationCredits to this forum member
« Reply #60 on: December 19, 2007, 03:31:50 AM »

one last question:
can I use a command line (DOS) to do the same ?

if so, what should be the formant ?
Logged
backtomind
Participant
*
Posts: 1

View Profile Give some DonationCredits to this forum member
« Reply #61 on: December 20, 2007, 06:49:59 AM »

Hi,
it is possible and if so what is the way to get the source code of the SpinZip ?
Logged
sm00ter
Participant
*
Posts: 1

View Profile Give some DonationCredits to this forum member
« Reply #62 on: February 26, 2008, 08:42:29 AM »

skwire,

This app is awesome!  I do think the password/encryption option would be nice!

Hey, while we are adding awesome features, how about cleaning up the original file/s once they are zipped?

For example:  If I request each demo.log file to be zipped to demo.zip, once it is zipped - delete the original demo.log file!

Woohoo!

Thanks,

sm00ter
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #63 on: February 26, 2008, 09:55:19 AM »

2. I am sending to this customers many games (hugh sized) which are more that 2 GB each.
Is it just me, or does this sound just a slight bit fishy?
Logged

- carpe noctem
J-Mac
Supporting Member
**
Posts: 2,854


see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #64 on: February 27, 2008, 12:45:28 AM »

2. I am sending to this customers many games (hugh sized) which are more that 2 GB each.
Is it just me, or does this sound just a slight bit fishy?

Or signing up just to ask for source code?
Logged

"I am getting so tired of slitting the throats of people who say that I am a violent psychopath."
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #65 on: March 10, 2008, 11:28:27 PM »

Hi,
it is possible and if so what is the way to get the source code of the SpinZip ?

I'm not interested in making the source available at this time.

skwire,

This app is awesome!  I do think the password/encryption option would be nice!

Hey, while we are adding awesome features, how about cleaning up the original file/s once they are zipped?

For example:  If I request each demo.log file to be zipped to demo.zip, once it is zipped - delete the original demo.log file!

Thanks for your kind words and apologies for the late reply.  Are you still interested in these additional features?
Logged

skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #66 on: March 10, 2008, 11:29:43 PM »

2. I am sending to this customers many games (hugh sized) which are more that 2 GB each.
Is it just me, or does this sound just a slight bit fishy?

Or signing up just to ask for source code?

Agreed.  I'm all for FOSS but it seems that some folks feel entitled to it these days.
Logged

ampedm
Participant
*
Posts: 5

View Profile Give some DonationCredits to this forum member
« Reply #67 on: March 18, 2008, 09:15:08 PM »

Hi,
it is possible and if so what is the way to get the source code of the SpinZip ?

I'm not interested in making the source available at this time.

skwire,

This app is awesome!  I do think the password/encryption option would be nice!

Hey, while we are adding awesome features, how about cleaning up the original file/s once they are zipped?

For example:  If I request each demo.log file to be zipped to demo.zip, once it is zipped - delete the original demo.log file!

Thanks for your kind words and apologies for the late reply.  Are you still interested in these additional features?

I found your program while looking for ones on google. I think the password feature is the only feature (besides comments... but thats IZARCCs fault smiley ) that this program needs to be more unique than winrar.

I have tried the following to put files in separate archives:

Winrar - need to pay,
PeaZip - uses more than one instance - a dirty trick
TugZip - No other feature beside simple archiving.

So i am sure everyone including me would love to see the next release have password support & if you got time, maybe a file browser to pick and choose files (CTRL + click thing)  + other easy to implement features with the IZARCC tool...

- Eagerly waiting!!
« Last Edit: March 18, 2008, 09:17:13 PM by ampedm » Logged
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #68 on: March 18, 2008, 10:57:06 PM »

I found your program while looking for ones on google. I think the password feature is the only feature (besides comments... but thats IZARCCs fault smiley ) that this program needs to be more unique than winrar.

I have tried the following to put files in separate archives:

Winrar - need to pay,
PeaZip - uses more than one instance - a dirty trick
TugZip - No other feature beside simple archiving.

So i am sure everyone including me would love to see the next release have password support & if you got time, maybe a file browser to pick and choose files (CTRL + click thing)  + other easy to implement features with the IZARCC tool...

- Eagerly waiting!!

I've already coded in password support but I'm hesitant to add file deletion support.  I'd really hate to screw that up, you know?  Though, on the other hand, if I delete to the recycle bin maybe it wouldn't be such a risk.  What are your thoughts on this?  The encryption idea will not be implemented since the IZarc command-line binary and the IZarc GUI version are the only things capable of opening those encrypted zips (I use the IZarc command-line binary as the backend for SpInZip). 

As for a file browser, I think that goes beyond the scope of this tool.  I've never intended for SpInZip to be anything close to a complete compression suite/app.  Like I mentioned before, SpInZip is nothing more than a fancy front-end for the IZarc command-line binary.  That being said, I do realise that Coding Snacks sometimes do take on a life of their own, hence, all the additional output options I've added to SpInZip.  I trust you can understand that.  =]

At any rate, here's v0.0.7.0 with the password support.  Please let me know how it works out for you.  Thanks.

http://www.autohotkey.net...pinzip-setup-v0.0.7.0.exe
Logged

ampedm
Participant
*
Posts: 5

View Profile Give some DonationCredits to this forum member
« Reply #69 on: March 18, 2008, 11:15:19 PM »

I found your program while looking for ones on google. I think the password feature is the only feature (besides comments... but thats IZARCCs fault smiley ) that this program needs to be more unique than winrar.

I have tried the following to put files in separate archives:

Winrar - need to pay,
PeaZip - uses more than one instance - a dirty trick
TugZip - No other feature beside simple archiving.

So i am sure everyone including me would love to see the next release have password support & if you got time, maybe a file browser to pick and choose files (CTRL + click thing)  + other easy to implement features with the IZARCC tool...

- Eagerly waiting!!

I've already coded in password support but I'm hesitant to add file deletion support.  I'd really hate to screw that up, you know?  Though, on the other hand, if I delete to the recycle bin maybe it wouldn't be such a risk.  What are your thoughts on this?  The encryption idea will not be implemented since the IZarc command-line binary and the IZarc GUI version are the only things capable of opening those encrypted zips (I use the IZarc command-line binary as the backend for SpInZip). 

As for a file browser, I think that goes beyond the scope of this tool.  I've never intended for SpInZip to be anything close to a complete compression suite/app.  Like I mentioned before, SpInZip is nothing more than a fancy front-end for the IZarc command-line binary.  That being said, I do realise that Coding Snacks sometimes do take on a life of their own, hence, all the additional output options I've added to SpInZip.  I trust you can understand that.  =]

At any rate, here's v0.0.7.0 with the password support.  Please let me know how it works out for you.  Thanks.

http://www.autohotkey.net...pinzip-setup-v0.0.7.0.exe

i found an error:

when i try to zip @ level 9 filter *.chm a file named "textbook_of_international_health__2nd_ed_0195132041.chm"

or

"Electrodiagnosis in Diseases of Nerve and Muscle - Principles and Practice.chm"

extra files mysteriously appear in the archive:

Logged
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #70 on: March 18, 2008, 11:33:10 PM »

i found an error:

when i try to zip @ level 9 filter *.chm a file named "textbook_of_international_health__2nd_ed_0195132041.chm"

or

"Electrodiagnosis in Diseases of Nerve and Muscle - Principles and Practice.chm"

extra files mysteriously appear in the archive:

Hmmm...  Please provide the following information:

1) What input & output paths are you using?
2) Is the "Scan subdirectories too?" checkbox checked?
3) Which output mode are you using along with its settings.

Thanks.
Logged

ampedm
Participant
*
Posts: 5

View Profile Give some DonationCredits to this forum member
« Reply #71 on: March 18, 2008, 11:36:34 PM »

i found an error:

when i try to zip @ level 9 filter *.chm a file named "textbook_of_international_health__2nd_ed_0195132041.chm"

or

"Electrodiagnosis in Diseases of Nerve and Muscle - Principles and Practice.chm"

extra files mysteriously appear in the archive:

Hmmm...  Please provide the following information:

1) What input & output paths are you using?
2) Is the "Scan subdirectories too?" checkbox checked?
3) Which output mode are you using along with its settings.

Thanks.

E:\TODO-test, (copy 1)
No
Last output mode with extension

with and without password
Logged
skwire
Charter Member
***
Posts: 4,036



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #72 on: March 19, 2008, 12:15:21 AM »

Gah...silly bug on my part.  Thanks for catching it.

Changelog:

[copy or print]
18 Mar 2008 - v0.0.8.0

    ! Fixed a pathing bug when using the "One file per zip" output mode.
      Thanks, ampedm.

12 Mar 2008 - v0.0.7.0

    + Added the ability to configure a password for generated zips.

New link here:  http://www.autohotkey.net...pinzip-setup-v0.0.8.0.exe

Apologies for the trouble and thanks!
« Last Edit: March 19, 2008, 12:19:15 AM by skwire » Logged

mouser
First Author
Administrator
*****
Posts: 33,292



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #73 on: March 19, 2008, 01:20:51 AM »

i love seeing this teamwork  thumbs up thumbs up thumbs up
Logged
ampedm
Participant
*
Posts: 5

View Profile Give some DonationCredits to this forum member
« Reply #74 on: March 19, 2008, 04:43:47 AM »

Works amazing might i suggest new features now smiley

- Making the priority of spinzip higher than izarcc and izarcc's lower so the spinzip window doesn't freeze.
- Cancel button of process.
- That's all i got...
Logged
Pages: Prev 1 2 [3] 4 5 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.089s | Server load: 0.14 ]