Main Area and Open Discussion > General Software Discussion

blanking file data while keeping folder-file structure

<< < (3/5) > >>

Shades:
Regarding LAN copy:
On my gigabit LAN Directory Opus reports speeds of 70 to 80MByte/sec it can sustain when copying sets of database dump files (each 4GByte in size). Because of this I don't even use any software to help and/or speed up copying anymore.  But I distinctly remember when I still was using a 100Mbit LAN, that the use of Teracopy improved LAN copying speeds significantly. However, this was years ago.

Regarding Internet copy:
I use robocopy (command-line) to transfer files from one continent to another. Actually I set up my own WebDAV server and use a script to start the transfer of files to my WebDAV server. Can't complain about the speeds. As a test I used Google drive and I was able to manually transfer a set of files in 10 minutes. Using my own WebDAV server I can transfer the same set of files fully automated in 12 minutes at night or around 15 minutes during the day.

If you use Google Drive, Dropbox, OneDrive etc. you have been using WebDAV. It allows you to transfer files over the HTTP protocol (port 80). Setting up a WebDAV server isn't too difficult in Linux (Apache2 webserver is what I use, but Ngix and Lighttpd web servers are also fully supported) and only slightly more complicated on IIS (Internet Information Server) in Windows Server.

If you are on a relatively simple 20Mbit/sec or higher connection (not bound by (too) much routing and/or firewall rules) between your computer at home and your current location you would be amazed how much data you can push over during one night of sleep. The higher the upload speed is of your connection at home, the better (if you transfer from home to your current location). If you can find out what the ip number of your current location is, you could even take a look at a (portable) piece of freeware called: CarotDAV. It allows you to set up a minimalistic WebDAV server on your current location and with the IP number of your current location you could try to initiate a transfer session on your home computer (TeamViewer or similar software) to your ad-hoc WebDAV server at your current location. Perhaps there is someone at your home location to start the transfer on your behalf, in case you don't have teamviewer.

For transferring stuff you don't want to park even for a moment on the common commercial cloud solutions, you can do way worse than using your own WebDAV server. WinSCP can also be used in combination with WebDAV. Also freeware, also portable and comes in both GUI-based and command-line application and is with regards to file transfers very powerful and reliable too. In power, complexity and reliability WinSCP and Robocopy are matched.

4wd:
The other RoboCopy offshoots?-Steven Avery (November 22, 2016, 09:24 PM)
--- End quote ---

RoboCopy derivatives that I mentioned here:

BTW, if you're going to use Robocopy where there's any chance that the files will already be opened then it will fail on those files, (it won't access open files).  You can get around that by either using:
1) ShadowSpawn (which is a replacement for Hobocopy below) which will create a VSS copy before running Robocopy, or;
2) Hobocopy which was a less featured version of Robocopy that could use the VSS, (written by the author of ShadowSpawn).
--- End quote ---

Although, ShadowSpawn is really just a pre-command to RoboCopy, (or any other command), that creates a VSS copy so that open files can be accessed.

Wikipedia also mentions a couple of other GUIs for RoboCopyw, like Easy RoboCopy which has been updated to cope with new args in the Win10 version.

Can you get 50 Gigabytes in hours rather than days with any of these tools like RichCopy/Robocopy?
--- End quote ---

File Transfer Time Calculator

eg. 50GB @ ~20Mb/s = ~6.3 hours

It all depends on whether you can saturate your connection and the fact that a lot of small file transfers will slow down the transfer due to increased overhead, (more negotiation). In which case it's probably better to Zip/7z the folder structure beforehand just using the store option instead of compression so you only have one big file to transfer at maximum throughput.

+1 for CarotDAV, although I use it as a single interface for accessing various cloud storage rather than as a WebDAV server.

IainB:
I'm not sure whether it can cope with leading spaces in file/folder names (I've never come across that before), but, using xplorer², the user can:

* Copy directory structures, without the data.
* List as a "Flat-file" display all the files in a directory structure, enabling the user to see, filter, sort and manipulate/edit/delete all files listed.
* Individually rename or mass rename files listed in a display, and the mass rename features are rather smart.
That might all be useful in your case.

tomos:
For straight sync, Bvckup was getting very good press here -- fast and accurate were the reports:
https://www.donationcoder.com/forum/index.php?topic=36388.0

haven't used it myself so dont know how it deals with any potential problem files.

Edit// I use Syncovery myself: subjectively very fast; again very accurate; partial file updating; can find moved files and move at destination rather than delete/recopy. It can ignore conflicts (there are a few options, let me look there), and send you an email if there's a problem (or if there's no problem as well).
But I havent really used it in this type of scenario so cant say a whole lot more about it.

tomos:
I use Syncovery myself: subjectively very fast; again very accurate; partial file updating; can find moved files and move at destination rather than delete/recopy. It can ignore conflicts (there are a few options, let me look there), and send you an email if there's a problem (or if there's no problem as well).
But I havent really used it in this type of scenario so cant say a whole lot more about it.
-tomos (November 23, 2016, 03:22 AM)
--- End quote ---

I actually cant find the email notification setting in Syncovery, I know it's there, I have it enabled for at least one profile :-/

Some relevant options:

blanking file data while keeping folder-file structure

blanking file data while keeping folder-file structure

see mouse arrow for how to access this dialogue (below):
blanking file data while keeping folder-file structure

Navigation

[0] Message Index

[#] Next page

[*] Previous page