Messages - vixay [ switch to compact view ]

Pages: prev • 1 2 3 4 [5] 6 7 8 9 10 ... 28 • next
21
General Software Discussion / Re: Recreate files but without content to target drive
« on: March 25, 2010, 08:39 AM »
Alright fellas,

2 hours spent trying to fix the script and here are the results for the python script!
[Select]
08:25:33 PM
08:25:46 PM
a grand total of 13 seconds!!!

Zipfile about 12 MB. with 54827 files (created a few files since last count)

Things I had to fix:
bugfixes
 modified time errors
 path length exceeded errors

resolution:
 #set time to unix epoch 0 = 'Thu Jan 01 07:00:00 1970'
so there will be some files with incorrect timestamps in the zipfile

I hope this helps everyone... though I still got some warnings while running the script... probably due to windows handling of files and or paths...
makeTreeZip.py:32: DeprecationWarning: struct integer overflow masking is deprecated
  zf.fp.write(zinfo.FileHeader())
makeTreeZip.py:32: DeprecationWarning: 'H' format requires 0 <= number <= 65535
  zf.fp.write(zinfo.FileHeader())
makeTreeZip.py:79: DeprecationWarning: struct integer overflow masking is deprecated
  zf.close()
makeTreeZip.py:79: DeprecationWarning: 'H' format requires 0 <= number <= 65535
  zf.close()

Here's the final code so far
Code: Python [Select]
  1. #!/usr/bin/python
  2. import os, sys, zipfile, optparse, zlib, fnmatch, time
  3.  
  4. optp=optparse.OptionParser(usage="%prog [options] <target-zip> <source-dir> [<source-dir...>]")
  5. optp.add_option("-x", help="exclude mask", action="append", dest="exclude", default=[])
  6. optp.add_option("-q", help="be quiet", action="store_true", dest="quiet", default=False)
  7. optp.add_option("-e", help="omit empty directories", action="store_true", dest="omit_empty", default=False)
  8. err=optp.error
  9.  
  10. options, args = optp.parse_args()
  11.  
  12. if len(args)<2:
  13.   optp.print_usage()
  14.   sys.exit(1)
  15.  
  16. zf=zipfile.ZipFile(args[0], "w")
  17.  
  18. def zfAddNullFile(zf, arcname, date_time, extattr=0):
  19.   """ Adapted from the method in zipfile """
  20.   arcname = os.path.normpath(os.path.splitdrive(arcname)[1])
  21.   arcname = arcname.lstrip(os.sep + (os.altsep or ""))
  22.   zinfo = zipfile.ZipInfo(arcname, date_time)
  23.   zinfo.external_attr = extattr
  24.   zinfo.compress_type = zf.compression
  25.   zinfo.file_size = 0
  26.   zinfo.flag_bits = 0x00
  27.   zinfo.header_offset = zf.fp.tell()    # Start of header bytes
  28.   zf._writecheck(zinfo)
  29.   zf._didModify = True
  30.   zinfo.CRC = CRC = zlib.crc32("")
  31.   zinfo.compress_size = 0
  32.   zinfo.file_size = 0
  33.   zf.fp.write(zinfo.FileHeader())
  34.   zf.filelist.append(zinfo)
  35.   zf.NameToInfo[zinfo.filename] = zinfo
  36.  
  37. def printFilename(f, msg=None):
  38.   if not options.quiet:
  39.     if msg:
  40.       print msg,
  41.    
  42.     try:
  43.       print f
  44.     except:
  45.       print f.encode("charmap", "replace")
  46.  
  47. for sourceDir in args[1:]:
  48.   try:
  49.     sourceDir=sourceDir.decode("latin-1")
  50.   except:
  51.     print sourceDir
  52.     print "Exception while trying to process directory"
  53.   for dp, dn, fn in os.walk(sourceDir):
  54.     if fn:
  55.       for f in fn:
  56.         f=os.path.join(dp, f)
  57.         ok=True
  58.         for xmask in options.exclude:
  59.           if fnmatch.fnmatch(f, xmask):
  60.             ok=False
  61.             break
  62.         if ok:
  63.           try:
  64.             mtime=time.localtime(os.stat(f).st_mtime)
  65.           except ValueError:
  66.             print "Error: Modified time out of range."
  67.             printFilename(f)
  68.             print os.stat(f).st_mtime
  69.             mtime=time.localtime(0) #set time to unix epoch 0 = 'Thu Jan 01 07:00:00 1970'
  70.           except WindowsError:
  71.             print "Error: Can't find file due to windows limitations."
  72.             printFilename(f)
  73.             mtime=time.localtime(0) #set time to unix epoch 0 = 'Thu Jan 01 07:00:00 1970'
  74.            
  75.           zfAddNullFile(zf, f, (mtime.tm_year, mtime.tm_mon, mtime.tm_mday, mtime.tm_hour, mtime.tm_min, mtime.tm_sec))
  76.     elif not options.omit_empty:
  77.       mtime=time.localtime(os.stat(dp).st_mtime)
  78.       #printFilename(dp, "(empty directory)")
  79.       zfAddNullFile(zf, dp, (mtime.tm_year, mtime.tm_mon, mtime.tm_mday, mtime.tm_hour, mtime.tm_min, mtime.tm_sec), 16)
  80.      
  81. zf.close()
  82. print "All Done."

I hope you all enjoy this. I'll attach the exe as well, though I can't make any guarantees for it.


22
General Software Discussion / Re: Recreate files but without content to target drive
« on: March 25, 2010, 07:23 AM »
Ok I needed to use the utility again and decided to do the speed comparison.

I ran it on A folder with
[Select]
67.8 GB (72,906,108,928 bytes)
54,824 Files, 5,364 Folders

Here's ZeroZipper
Start - 06:29:44 PM
End - 06:36:41 PM
It took 7 minutes!! Out of which 5 minutes was spent just recreating the 0-byte structure! 1 minute to delete files and <1 to create the zip.
So this method is definitely not good for large folders.
And I think I found a bug. For the zip I didn't put a path just a name "zz.zip", the zip was created in the temp correctly but then it didn't move it to the same location as the zerozipper... it was strange... and the zip was then deleted! so I didn't get any output after waiting for so long!

I then had high hopes for the python version, as it would avoid the slow Disk I/O. So
 i finally used the py2exe and converted the code to an exe and tried that (rather than the script just to make a fair comparison).
It died again on unicode filenames :(. I did use the new version too this time!
Traceback (most recent call last):
  File "makeTreeZip.py", line 61, in <module>
    mtime=time.localtime(os.stat(f).st_mtime)
WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect: 'J:\\Share\\Categorized Appz\\Compression Utilities\\Just Extractor\\??-readme.txt'

Can someone help with this problem? It seems like the filenames which contain unrecognizable characters are converted to ?? and then we we use them again it crashes... how do we retain the filenames in the original format? regardless of whether it contains Legible/illegible unicode?

After this i tried running the script directly and it seemed to work but then
I encountered another mysterious error
Traceback (most recent call last):
  File "makeTreeZip.py", line 63, in <module>
    mtime=time.localtime(os.stat(f).st_mtime)
ValueError: (22, 'Invalid argument')

I tried debugging this the value for st_mtime was 1189361328.0 i converted that using time.localtime() and it worked, so i can't figure out exactly what is causing this error.

Actually the file in question have a modified date of
Tuesday, September 25, 3427, 10:40:30 PM
so i can see how python would choke on that. how can i resolve this issue?

23
Living Room / Re: Desktop Icons
« on: March 08, 2010, 11:55 PM »
some autohotkey code i use for this functionality
[Select]
;function to return toggled values
;first call returns true
toggle() {
 static t
 t := !t
 Return, t
}
EnableDisableDesktopIcons()
{
  ;relies on having DetectHiddenWindows OFF
  if toggle()
    WinShow,Program Manager
  Else
    WinHide,Program Manager
}


#`::EnableDisableDesktopIcons() ;;Enable/disable desktop icons (shortcut key Win+`)

Doesn't work with fences installed though, but you can then use fences, ESC key to toggle it.

24
General Software Discussion / Re: virtual display utility
« on: March 08, 2010, 11:48 PM »
How about setting a resolution higher than what is allowed?

Sometimes this is what happens: You see only a part of the screen (i.e. your native resolution) but if you move your mouse in any direction, the screen scrolls and shows that part... and voila your screen acts like a window for a larger screen!

This is happened to me when playing games ...etc and the switch back to the desktop was not made correctly, i don't know if it's possible to induce this behavior!

25
Living Room / Re: Blog Essay: In Praise of Online Obscurity
« on: March 08, 2010, 11:02 PM »
http://en.wikipedia.org/wiki/Dunbar%27s_number

The theoretical hard limit on how many social connections we can actively maintain.

About 150.

The discussion reminded me about this.

Pages: prev • 1 2 3 4 [5] 6 7 8 9 10 ... 28 • next
Go to full version