Main Area and Open Discussion > General Software Discussion

File synchronization: moving away from incremental backup (HELP!)

<< < (2/13) > >>

f0dder:
Ralf Maximus and tomos: both SyncBack and SuperFlexible seem to be run by a scheduler, instead of doing transparent mirroring... while this could be enough if it's 100% invisible in the background, it still requires checking each file in the "monitored" hierarchy for changes, instead of using filesystem event notifications?

Reading the versus-thread was a good idea though, I will be checking into FileHasmter, MirrorFolder and Second Copy, as those three might be closer to what I need.

Remember, I just need to sync/mirror userdata to \\server\backup\<username>, I don't need any incremental backups/versioning, ftp support, compression etc - all that will be handled by another piece of software responsible for backing up the server.

yksyks: thanks for the link, but we've already settled on a remote backup host (we're moving to fiber internet, and that company also offers remote backup). We need around 100-200gigs of space (and more every year), so transferring the baseline backup will be done by techs from the company, instead of taking a lot of time with a limited 5mbit upload :)

tomos:
[SFFS] still requires checking each file in the "monitored" hierarchy for changes, instead of using filesystem event notifications?-f0dder (November 09, 2007, 07:24 AM)
--- End quote ---

I believe so

nosh:
I'm pretty certain Second Copy checks each file too coz they warn you of slowdowns when you enable realtime mirroring.

Ralf Maximus:
Ralf Maximus and tomos: both SyncBack and SuperFlexible seem to be run by a scheduler, instead of doing transparent mirroring... while this could be enough if it's 100% invisible in the background, it still requires checking each file in the "monitored" hierarchy for changes, instead of using filesystem event notifications?

--- End quote ---

Aye, tis true.  However I am extremely pleased at how fast SyncBack performs its checks.  I routinely synchronize network folders containing dozens of gigabytes, and if there are no changes it takes less than a minute.  Massive changes, of course, incur the time required to copy the files -- standard network traffic, which could take many minutes.

If the file's in use (or becomes in-use) during a copy, SyncBack notes the event in the log and moves on without hassling the user.  This is a configuration setting.

As far as background processing... I believe it can be configured to run silently, with no visible GUI.  I often call SyncBack from .bat files using its command-line interface, and those events generate nothing visible, not even a tray icon.  And it throttles its CPU demand to prevent taking over the user's descktop experience; aside from a chattering disk drive they won't even know it's running.

Not bad for a freebie.

f0dder:
I've checked out MirrorFolder a bit, and it seems very very very promising.

As far as I've been able to tell, the best the other applications can offer is detecting modified files via filesystem events - this has two problems. One is that Windows' fsevents only tell you that "there's a change somewhere in the tree you're monitoring", not WHAT has changed - which means you have to scan the entire tree upon notification (unless you resort to dirty tricks like reading the NTFS journal). The other problem is that you have to synchronize the entire file, even if only part of it has changed (probably not too big a problem for my uses, but still...)

MirrorFolder is cool, though. It can install a filter driver that monitors changes, so only the changes are synced - ie., changing 10 bytes in a 10gig file only pushes those 10 bytes to your sync destination. And it even works with network shares (couldn't find a way to set user credentials, but mapping at network drive worked like a charm). The driver is 73kb and the server is 118kb, so it's not exactly bloatware either.

I will have to do some further testing of this, but MirrorFolder seems like a natural winner for my purposes, if it doesn't impact performance too much, and (of course!) is stable. But I expect the filter driver approach to have pretty decent performance. The only nag I have about MF is that they call the filter drive method for "RAID-1", which is wrong and misleading and doesn't give the technique enough credit :)

Still, if anybody has other suggestions, keep 'em coming!

Navigation

[0] Message Index

[#] Next page

[*] Previous page