topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Monday March 18, 2024, 10:48 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: Recreate files but without content to target drive  (Read 56709 times)

vixay

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 140
  • ViXaY
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #25 on: March 25, 2010, 08:39 AM »
Alright fellas,

2 hours spent trying to fix the script and here are the results for the python script!
08:25:33 PM
08:25:46 PM
a grand total of 13 seconds!!!

Zipfile about 12 MB. with 54827 files (created a few files since last count)

Things I had to fix:
bugfixes
 modified time errors
 path length exceeded errors

resolution:
 #set time to unix epoch 0 = 'Thu Jan 01 07:00:00 1970'
so there will be some files with incorrect timestamps in the zipfile

I hope this helps everyone... though I still got some warnings while running the script... probably due to windows handling of files and or paths...
makeTreeZip.py:32: DeprecationWarning: struct integer overflow masking is deprecated
  zf.fp.write(zinfo.FileHeader())
makeTreeZip.py:32: DeprecationWarning: 'H' format requires 0 <= number <= 65535
  zf.fp.write(zinfo.FileHeader())
makeTreeZip.py:79: DeprecationWarning: struct integer overflow masking is deprecated
  zf.close()
makeTreeZip.py:79: DeprecationWarning: 'H' format requires 0 <= number <= 65535
  zf.close()

Here's the final code so far
Code: Python [Select]
  1. #!/usr/bin/python
  2. import os, sys, zipfile, optparse, zlib, fnmatch, time
  3.  
  4. optp=optparse.OptionParser(usage="%prog [options] <target-zip> <source-dir> [<source-dir...>]")
  5. optp.add_option("-x", help="exclude mask", action="append", dest="exclude", default=[])
  6. optp.add_option("-q", help="be quiet", action="store_true", dest="quiet", default=False)
  7. optp.add_option("-e", help="omit empty directories", action="store_true", dest="omit_empty", default=False)
  8. err=optp.error
  9.  
  10. options, args = optp.parse_args()
  11.  
  12. if len(args)<2:
  13.   optp.print_usage()
  14.   sys.exit(1)
  15.  
  16. zf=zipfile.ZipFile(args[0], "w")
  17.  
  18. def zfAddNullFile(zf, arcname, date_time, extattr=0):
  19.   """ Adapted from the method in zipfile """
  20.   arcname = os.path.normpath(os.path.splitdrive(arcname)[1])
  21.   arcname = arcname.lstrip(os.sep + (os.altsep or ""))
  22.   zinfo = zipfile.ZipInfo(arcname, date_time)
  23.   zinfo.external_attr = extattr
  24.   zinfo.compress_type = zf.compression
  25.   zinfo.file_size = 0
  26.   zinfo.flag_bits = 0x00
  27.   zinfo.header_offset = zf.fp.tell()    # Start of header bytes
  28.   zf._writecheck(zinfo)
  29.   zf._didModify = True
  30.   zinfo.CRC = CRC = zlib.crc32("")
  31.   zinfo.compress_size = 0
  32.   zinfo.file_size = 0
  33.   zf.fp.write(zinfo.FileHeader())
  34.   zf.filelist.append(zinfo)
  35.   zf.NameToInfo[zinfo.filename] = zinfo
  36.  
  37. def printFilename(f, msg=None):
  38.   if not options.quiet:
  39.     if msg:
  40.       print msg,
  41.    
  42.     try:
  43.       print f
  44.     except:
  45.       print f.encode("charmap", "replace")
  46.  
  47. for sourceDir in args[1:]:
  48.   try:
  49.     sourceDir=sourceDir.decode("latin-1")
  50.   except:
  51.     print sourceDir
  52.     print "Exception while trying to process directory"
  53.   for dp, dn, fn in os.walk(sourceDir):
  54.     if fn:
  55.       for f in fn:
  56.         f=os.path.join(dp, f)
  57.         ok=True
  58.         for xmask in options.exclude:
  59.           if fnmatch.fnmatch(f, xmask):
  60.             ok=False
  61.             break
  62.         if ok:
  63.           try:
  64.             mtime=time.localtime(os.stat(f).st_mtime)
  65.           except ValueError:
  66.             print "Error: Modified time out of range."
  67.             printFilename(f)
  68.             print os.stat(f).st_mtime
  69.             mtime=time.localtime(0) #set time to unix epoch 0 = 'Thu Jan 01 07:00:00 1970'
  70.           except WindowsError:
  71.             print "Error: Can't find file due to windows limitations."
  72.             printFilename(f)
  73.             mtime=time.localtime(0) #set time to unix epoch 0 = 'Thu Jan 01 07:00:00 1970'
  74.            
  75.           zfAddNullFile(zf, f, (mtime.tm_year, mtime.tm_mon, mtime.tm_mday, mtime.tm_hour, mtime.tm_min, mtime.tm_sec))
  76.     elif not options.omit_empty:
  77.       mtime=time.localtime(os.stat(dp).st_mtime)
  78.       #printFilename(dp, "(empty directory)")
  79.       zfAddNullFile(zf, dp, (mtime.tm_year, mtime.tm_mon, mtime.tm_mday, mtime.tm_hour, mtime.tm_min, mtime.tm_sec), 16)
  80.      
  81. zf.close()
  82. print "All Done."

I hope you all enjoy this. I'll attach the exe as well, though I can't make any guarantees for it.

"Drunk on the Nectar of Life!" -me
« Last Edit: January 03, 2013, 07:04 AM by vixay, Reason: Added corrections to script, and shebang for unix systems, updated exe and added script to zip »

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #26 on: March 25, 2010, 02:53 PM »
Nice job, vixay.   :D

feerlessleadr

  • Participant
  • Joined in 2010
  • *
  • default avatar
  • Posts: 1
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #27 on: April 23, 2010, 09:54 AM »
Hey guys, love the software in here.  I am currently using zero zipper to create the 0 byte files but I was wondering if there were any command line options that I could use so that I could utilize zero zipper in a batch file.

I tried to use vixay's exe, but i'm not sure I completely understand how to use it.

Thanks for your help

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #28 on: April 25, 2010, 10:15 PM »
You should be able to use vixay's exe for what you want to do on the commandline like this:

makeTreeZip.exe target-zip-file.zip mydir1 mydir2

Let us know how that works out for you.

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #29 on: October 24, 2011, 03:34 PM »
Website | Download
v1.0.5 - 2011-10-24
    + Zero Zipper now preserves both the modified and created timestamps for
      files and folders within the zipfile.  (Thanks, Wayne)
    + Added drag-n-drop to the two edit fields.  (Thanks, Wayne)
    ! Fixed 120 DPI display issues.  (Thanks, Wayne)
    ! Fixed a parsing issue where not all files would get cloned into the zip.

lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #30 on: February 20, 2012, 03:03 AM »
I have been looking for something like Zero zipper for a long time.  My goal being to create catalogs of offline drives\folders (including files!) without needing any special software to read the "catalog" - just any filemanager.   I have run into a problem though where sometimes a few dummy files are not included in the zip.  I think the problem is the 255 character limit in Windows.  Since Zero Zipper uses in my case "C:\Documents and Settings\username\Local Settings\Temp\$$$$$20120220035412_ZeroZipper_temp".

Could you add an option so I can choose a temp folder located on the C root drive?

thank you!

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,610
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #31 on: February 20, 2012, 03:13 AM »
Welcome to DonationCoder, lastgasp,

so I can choose a temp folder located on the C root drive?
You can normally do that by setting the TMP (and sometimes TEMP) environment variable to the desired location (f.e. set TMP=C:\TMP), just before starting the application. A simple batchfile could be a way to do that.

lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #32 on: February 20, 2012, 12:18 PM »
I changed the temp file as suggested and got a much better result.  Sadly I still lost some files.  The temp folder looks like this "C:\TEMP\$$$$$20120220122310_ZeroZipper_temp" .  Various test with zero zipper seem to confirm that the problem is the 255 limit.  Perhaps if "C:\TEMP\$$$$$20120220122310_ZeroZipper_temp" could be shorten to something like "C:\TEMP\Zero_temp" or even shorter it would would not skip any files.

Also I would like to use Zero Zipper on other peoples computers and would not be comfortable changing their temp file location folder.  I feel like I'm asking too much but if Zero Zipper could be revised to deal with my issues it would very much change my life.  I took a USB flash drive and formatted it to NTFS which allows Everything for Windows from voidtools.com to read the USN Journal and thus created a "dummy drive". I'm hoping to put several "dummy drives" on a single USB flash drive.  Since both Everything for Windows and Zero Zipper are portable I dream of having a complete catalog of all my drive in one USB flash.

thanks!


MilesAhead

  • Supporting Member
  • Joined in 2009
  • **
  • Posts: 7,736
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #33 on: February 20, 2012, 09:05 PM »
Also I would like to use Zero Zipper on other peoples computers and would not be comfortable changing their temp file location folder.

When you set an environment variable on the command line, that only lasts until you close the prompt.  Try it yourself. Open a command prompt. Type
set temp=c:\temp

then type
echo %temp%

Then exit the command prompt. Open another command prompt and type
echo %temp%

It should show the original temp directory, not c:\temp.

I just did it on Windows 7 and it still works as expected.

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #34 on: February 22, 2012, 07:41 PM »
Hi, lastgasp, and welcome to the site.  Please give this test build a try:  Zero Zipper v1.0.5.2

This version has a customisable temp folder setting but let me explain how it works.  You specify the folder to use and Zero Zipper will then create a folder with a really short name (basically, the first number from one that doesn't exist) and use that to create the zero-byte structure.  So, if you specify something like c:\tmp, the actual folder that will be used will be something like c:\tmp\1.  This allows me to work within a new, empty folder and easily delete it when finished zipping it.  At any rate, give this version a shot and let me know how it goes.  Thanks and sorry for the late reply.
« Last Edit: February 22, 2012, 07:47 PM by skwire »

lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #35 on: February 23, 2012, 12:39 PM »
The customisable temp folder setting works but strangely the resulting temp.zip is always empty!  I tried different settings on four computers all resulting in an empty temp.zip.  The only exception is if I leave the temp folder setting blank\default then it creates the temp.zip correctly.  But of course this temp.zip does not contain all the files because of the 255 limit.   :(

BTW thanks for "Files 2 Folder" I love that!



skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #36 on: February 23, 2012, 04:06 PM »
What are you using for the temp folder value?

lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #37 on: February 23, 2012, 07:50 PM »
Besides "leave blank for default" I has used a couple of dozen variations including root drives.  I have tried putting temp folder on usb flash, ssd sata, and in windows 7 a RAM disk.  Except for "leave blank for default" option all produce an empty temp.zip.  I have tried changing the name from temp.zip to other names.  I have tried to put the destination zip in various places with no luck.

I have opened the the 1kb temp.zip in notepad and get this some that looks like this  PK |-   not exactly though because I ca't get the last 2 characters to copy and paste.

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #38 on: February 25, 2012, 12:13 PM »
Try this build, please:  Zero Zipper v1.0.5.5

lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #39 on: February 25, 2012, 10:11 PM »
The temp.zip is working correctly now.  But I found that sometimes the file count inside the temp.zip was wrong.  Some files were still being omitted from the temp.zip. This was a mystery that I wanted to solve so I tested many folders.  Customizing the temp folder location eliminated file name length as a problem. Using Windows 7 windows explorer I found files with question marks inside of a black diamond instead of a recognizable character\number.   I searched "question mark inside of a black diamond" and I found something called a Unicode Character 'REPLACEMENT CHARACTER' .

wikipedia entry is here http://en.wikipedia....eplacement_character

I mostly use a Windows Xp machine so it took a while to figure it out.


lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #40 on: February 26, 2012, 12:33 AM »
I found a file displayed in Windows Explorer as "Rubйn Gonzбlez"  that is also skipped by zero zipper in the temp.zip.  "Rubйn Gonzбlez" should be "Rubén González" and when I correct the misspelling in Windows Explorer Zero Zipper includes it in the temp.zip with no problems.

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #41 on: February 26, 2012, 10:05 AM »
Try this build, please:  Zero Zipper v1.0.5.6

This one should have basic Unicode support.

lastgasp

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 7
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #42 on: February 29, 2012, 05:39 PM »
Thank you everything works great now!  I dropped 10 bucks in your tip jar at paypal.

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #43 on: February 29, 2012, 07:18 PM »
Thank you everything works great now!  I dropped 10 bucks in your tip jar at paypal.

Great to hear everything is working as requested.  Thanks very much for the donation.  I'll promote this build to v1.0.6 and release it.   :D

SkyFrontier

  • Participant
  • Joined in 2011
  • *
  • Posts: 2
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #44 on: December 09, 2012, 07:10 AM »
Hi, people!
Optionally it could preserve original system attributes, too.
Is it possible to have a tick box to NOT preserve system/timestamps? (separately...)
One last thing would be having no zip at all, just the cloned directories and files at target location. Haven't tested thoroughly yet but: can target be a subfolder of source and still there will be no overlap? (initial tests say it's ok, but the real thin would be with no zip creation, I think)
Thanks!

EDIT: yes, it would be nice to have a companion txt report on target folder/along with zip file or flat clone for comparison with previous snapshots, like Brycestro said.
« Last Edit: December 09, 2012, 07:44 AM by SkyFrontier »

Biffle

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 18
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #45 on: August 16, 2016, 04:57 AM »
Can Zero Zipper do this "Recreate files but without content to target drive" without zipping the files, but just create them in the target (so one would not to have to unzip them each time)?
Windows 10 Home, 64bit

vixay

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 140
  • ViXaY
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #46 on: August 16, 2016, 05:30 AM »
You could replace the commands to add the file to zip in the python script (see below) with your own python commands to create the file in the destination folder?
Replace these lines
zfAddNullFile(zf, f, (mtime.tm_year, mtime.tm_mon, mtime.tm_mday, mtime.tm_hour, mtime.tm_min, mtime.tm_sec))

zfAddNullFile(zf, dp, (mtime.tm_year, mtime.tm_mon, mtime.tm_mday, mtime.tm_hour, mtime.tm_min, mtime.tm_sec), 16)
with where you prepend your destination to both f & dp. e.g. f = target + f
open(f, "a").close()
os.mkdir(dp)
Though you would have to google on setting the timestamps accurately. see here   and here

or use zerozipper by skwire and choose your temp directory to be the destination where you want the structure, but to prevent deletion you would have to ask skwire for the changes or the code
"Drunk on the Nectar of Life!" -me

Biffle

  • Participant
  • Joined in 2012
  • *
  • default avatar
  • Posts: 18
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #47 on: August 16, 2016, 05:40 AM »
Thank you, vixay,

Looks complicated. Hmmm, where could I find this python script?

Many thanks again
Windows 10 Home, 64bit

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,640
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #48 on: August 16, 2016, 09:31 PM »
Looks complicated. Hmmm, where could I find this python script?

Just use RoboCopy (included with Windows):

robocopy <source> <dest> /CREATE /E /np /nfl /ndl /njh /njs

It'll do it in a few seconds.

vixay

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 140
  • ViXaY
    • View Profile
    • Donate to Member
Re: Recreate files but without content to target drive
« Reply #49 on: August 16, 2016, 11:20 PM »
Thank you, vixay,

Looks complicated. Hmmm, where could I find this python script?

Many thanks again

Here
"Drunk on the Nectar of Life!" -me