Messages

1

General Software Discussion / Re: teracopy: copy your files faster

« on: January 16, 2009, 09:32 PM »

I understand what your saying, I'm just saying if it will stop/not copy if the destination already has the file wanting to be copied, you could allow a setting/switch make it test both anyways just to see if its the same

-Gergith (January 16, 2009, 07:55 PM)

That's exactly how the date/time and file size compare options work now. How often are there files that you want to copy that have the exact same date/time and file sizes? In quite a few years, I have never needed this ability, but others might...anyone?

2

General Software Discussion / Re: teracopy: copy your files faster

« on: January 16, 2009, 07:53 PM »

Why did you pick that one? (im asking from lack of knowledge, its not a trap :p)

-Gergith (January 16, 2009, 07:41 PM)

Yes, just SHA1. I used this because it was easily accessible in C# in .NET. It seems pretty quick (in my very limited testing). Since we aren't concerned about collisions (other files generating the same hash), I figured it would work. If there are other things that would work better, I wouldn't mind hearing about them...

You're right, I could compare before copying, but then if the files were different, it would be reading both files an extra time. I figured that the main goal was to make sure that the copy McTool did worked properly.

3

General Software Discussion / Re: teracopy: copy your files faster

« on: January 16, 2009, 07:27 PM »

Of the ones mentioned so far, can you guys tell me which ones for certain can do a CRC, or any sort of Hash check on source and destination files?

-Gergith (January 16, 2009, 02:03 PM)

I added this ability to McTool today. If it fails to compare, it will retry (up to the specified number of times) until it gets it right.

With verbose logging turned on you can see the SHA1 hash in the log for each file, otherwise, it just reports when a compare fails.

4

General Software Discussion / Re: teracopy: copy your files faster

« on: January 08, 2009, 12:04 AM »

Hmm, what's the point of doing it multi-threaded? Isn't this just going to thrash the disk drives seek requests?

-f0dder (January 07, 2009, 10:21 PM)

On drives with a single spindle, you're right to suspect minimal or even negative improvement, yet most of the time with relatively small files (perhaps even as large as, say, files in an MP3 collection), you can still gain a performance advantage by using a small number of threads (2-5 has worked well for me in the past) on single drives.

Some things that affect whether multi-threading will improve copy speeds:

• The sizes of the files.
• The availability of NCQ on the drives (both source and destination).
• The sizes of the internal memory caches on the drives.
• The number of spindles that make up the volumes (e.g. a RAID array)
• Other hardware caching of data on the drives (NAS controllers, etc.)

On high-end systems (like these) I often see 10 times the throughput using multiple threads (whether using McTool or multiple instances of RoboCopy) when copying smallish files from one system to another. I have seen throughput speeds of over 90 MB/s in a single instance of the tool.

I don't know enough about SMB, CIFS, or Completion Ports to talk intelligently about them, but in developing this application, I have run into enough information about them to think it would be worth learning more...

5

General Software Discussion / New multi-threaded copy program

« on: January 07, 2009, 05:15 PM »

Anyone want to try out yet another copy program? It is multi-threaded and was designed to improve copy speeds on high-end network storage devices. It's mostly for use with lots (millions!) of small files, but works in other situations as well. Not a big deal - just thought I'd share if anyone can use such a thing...

McTool - web page

• Multi-threaded
• Drag & drop
• Directory synchronization
• E-mail report when finished
• Save parameter files
• Run parameter files from command line
• Log actions to file
• Retry on error
• Wildcard (Regular Expression) file matching
• Pause/resume

6

General Software Discussion / Re: Why is it so hard to find a decent image organizer?

« on: March 01, 2007, 08:55 PM »

I found a new program - FotoAlbum Pro from www.fototime.com.

It lacks some of the more power user features but ts very fast and handles metadata and queries very nicely. I emailed the developers with some suggestions and received a prompt reply which is a good sign.

-MrCrispy (March 01, 2007, 01:03 AM)

I've been using FotoAlbum for a couple of years now and I've loved it!  I've made suggestions, and they've implemented them - many times.  Most recently, I exchanged 20 e-mails with the head developer in order to fix a minor issue they had with the display of thumbnails in Firefox on their companion website (www.fototime.com).

It handles the meta information quite well for me.  I will NEVER again use something that uses a separate database to save meta information about my images without also maintaining the information in the images themselves - I've lost information that way too many times.

I have about 25,000 images in it now and it hasn't slowed down at all since I had 10,000 when I first started using it.