Main Area and Open Discussion > General Software Discussion
File synchronization: moving away from incremental backup (HELP!)
johnfdeluca:
I use Tivoli Continuous Data Protection (again...disclaimer, I work for IBM). Note...this is different than Tivoli Storage Manager. It is $42 and I have been pleased with it. I use it to do insta-synching of files (you might call this mirroring) to a network NAS (which does RAID). It runs in the background, seems to take small resources to run and auto backs up each file as soon as it is saved or updated. I particularly like:
* Set it and Forget it - I never really need to check on it. The only time I am reminded that it is running is when it cannot connect to the network drive
* Continuously protects versions of files to allow customers choice of recovery points
* Destination is not in propriatary format.....same format
* Offers to only send the portion of the file that has changed vs. entire file again
* Encrypted and/or Compressed optionshttp://www-306.ibm.com/software/tivoli/products/continuous-data-protection/
Free Trial here: http://www-306.ibm.com/software/tivoli/resource-center/storage/cdp.jsp
Flash Demo here: http://www-306.ibm.com/software/tivoli/library/demos/cdp.html
All that said, I'm not sure I'd shell out $42 for it with free options out there but there is something to be said for Big Blue behind the scenes.
General Settings Screenshot
File synchronization: moving away from incremental backup (HELP!)
Files to Protect Screenshot
File synchronization: moving away from incremental backup (HELP!)
Email Protection Screenshot
File synchronization: moving away from incremental backup (HELP!)
Remote Storage Screenshot
File synchronization: moving away from incremental backup (HELP!)
Advanced Settings Screenshot
File synchronization: moving away from incremental backup (HELP!)
f0dder:
Thanks, John...
Offers to only send the portion of the file that has changed vs. entire file again
--- End quote ---
Do you have any idea if it does this with the help of a filter driver, or (like SFFS/whatever) by scanning the file for changes?
johnfdeluca:
Do you have any idea if it does this with the help of a filter driver, or (like SFFS/whatever) by scanning the file for changes?
-f0dder (November 30, 2007, 06:32 AM)
--- End quote ---
I don't know but here is the relevant portion of the Help File:
Use sub-file copy check box
Set sub-file copy for remote backup copies.
Initially, an entire file is copied to the storage areas. When sub-file copy is turned on, and when the file changes, only the changed information is copied to the storage area. The sub-file copies are saved as separate files on the remote storage.
Sub-file copy can significantly reduce the amount of network traffic. However, sub-file copy consumes more processing resource on your computer. The default setting is to use sub-file copy for files larger than 50 MB. If you need to conserve more network resources, you can reduce the size setting so sub-file copy will be used on even smaller files.
Check the box to turn on sub-file copy. In the Use sub-file copy for files larger than: field, specify the file size threshold for using sub-file copy. For files larger than this size, only the changed information is copied to the storage area.
--- End quote ---
And from the Redbook:
Sub-file copy and versioning
If you configure Tivoli Continuous Data Protection for Files to use sub-file copy, it will create files that contain each file's delta information in your designated storage area. Those files have an FPdelta file name suffix.
To test the sub-file copy functionality the file Sub-file example.txt was modified. This file is larger than 3 MB. Figure 4-23 shows the replicated directory in our HTTP Server Paris. As you can see, we have the original file with 3,208 KB and a delta file with only the modified characters which is only 1 KB. Note that this feature is supported only with remote storage.
To track versions of a file, Tivoli Continuous Data Protection for Files adds a version suffix to the file name of the backup copy. On remote storage, all backup copies contain a version suffix. When a file is deleted on your computer, Tivoli Continuous Data Protection for Files will add a version identifier to the file name of the most recent backup copy on the remote storage.
The most recent backup copy of a file is the active backup copy. Older backup copies of that file are inactive backup copies. Tivoli Continuous Data Protection for Files will delete inactive backup copies of a file first if the storage space limit is reached.
A file that is protected by schedule could change several times during the schedule interval. Only the last version of the file prior to the end of the schedule will be backed up. A continuously protected file is backed up after every change.
--- End quote ---
And finally, from the Tivoli Storage Manager Redbook:
Adaptive subfile backup
Traditional storage management techniques have focused on protecting systems that stay in the same place and are always connected to the network. Today, many workers are mobile and/or remote, and keep critical data assets on their mobile computers (laptops) and other wireless devices. Storage administrators need new ways to protect mobile and remote computers with limited access to the infrastructure that serves the rest of the company. Some limitations include being attached to the corporate network with reduced bandwidth, limited connect time, and minimal assistance to perform the backup.
This limited access both increases the criticality of storage management services and limits the applicability of traditional methods and policies. Tivoli Storage Manager helps resolve these problems with its adaptive subfile backup feature, which reduces the amount of data transferred while backing up changed files.
This features enables the backup-archive client to back up only the changed portion of a file, either on byte level or on block level, instead of transferring the whole file to the server every time. The changed file portion is backed up as a differential backup relative to the last complete backup of the file (base or reference file) and it is called delta file. All changes since the last complete backup of the file are included in this delta file. In the case of a restore, this allows for the restore of the whole file by restoring only two sub-file components, one delta file, and the last complete backup of the whole file, the base file.
The decision to base the differential on byte level or block level will be made at the backup of the base file, and it depends on the size of the file. Subfile backup technology is not used for very small files (less than 1 KB in size) or for very large files (bigger than 2GB). If the delta file size exceeds 60 percent of the base file at the last sub-file backup, a new base file will be transferred.
The adaptive subfile backup, as well as the restore of a file consisting of a base file and the delta file, is completely transparent to you as a user. All necessary file data separations or reconstructions happen under-the-covers of the backup-archive client. Also, all other Tivoli Storage Manager features, such as policy management or fault-tolerant backup and restore, still fully apply. Adaptive subfile backup is used for incremental as well as for selective backup. It is aware of multithreading and will work together with client data compression and encryption.
--- End quote ---
nitrix-ud:
just my 0.05 cent
and please bear with me if it's already been said
by far, the best solution i've seen for folder sync is FolderShare (http://www.foldershare.com)
there are however some problems (which can be deal breakers !)
* 10000 file limit per folder (you can sync up to 10 folder)
* sometimes the folder share server does not work, and in that case the sync does not work...
for my backup need i use SyncbackSE (which is very good)
well in any case FolderShare is the best app i know along with autohotkey and FARR ;)
so you may give it a try, if the limitations are ok with you
i use it for collaborative work and it's just such a productivity boost...
i could not live without it
FolderShare is the only app i know that is really unobtrusive and real-time...
from what i recall mirror folder is real-time until the destination is offline... (am i wrong here ?)
I've also tried (but not fully tested) PowerFolder (similar to FolderShare)
Cheers, Nitrix
johnfdeluca:
10000 file limit per folder (you can sync up to 10 folder)
-nitrix-ud (November 30, 2007, 10:51 AM)
--- End quote ---
10 folder limit seems pretty limiting.....Is this a free version limit?
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version