Program to copy contents of a cloud drive to a local drive -slowly

[questorfla]

I guess that is the best way to put it.  I have a need to get the contents of our CLOUD storage onto a local system.  This is a pretty big pile of files and while I can sit here and manually do this one folder at a time, I would rather not as there are several hundred folders.  My first though was to simply run a SYNC program but I really don't need a Sync, other than maybe as a "one-way" thing.  There is nothing to Upload or sync back to.  I just need to "back-up" the Cloud.

I have used a couple like 'All-way Sync' before and maybe that is still the best way to go.  Some of the Sync programs tend to get Overly in a hurry which in this case I am not and preserving enough bandwidth to function would be a main concern. 

I need to download about 60GB of files.. Slowly!  Which in most cases is not what people are after. Using Windows, for instance, is hopelessly chaotic.  At least the GUI part.   If I try to drag and drop the whole thing at once it always crashes.  Folder by folder works but is getting tedious. 

Maybe a program that would drag and 'drop-to-copy' one folder at a time and do the next when the first is done?  Or a program that gave a way to limit the number of files it would try to copy at once?  Some way to limit the amount of computer overhead and bandwidth use so that everything copies... eventually.. and the system can run normally all the while albeit a little slower? 

The Cloud drive is mapped as a network drive and I have already begun the local folder copy of it but ran into these issues and thought I would see if anyone else has ever tried to "back-up" out of a cloud before and knew a better way.    Every time I Google Cloud Backup .. Well I am sure you know what I find,  Everyone going the Other Way

[worstje]

Try looking at file copying software like TeraCopy, TotalCopy or similar. I used these over a decade ago, so they may not be the best option anymore. You can just copy files with them as usual, but besides some retry-stuff in case of dropping connections, there's also a speed slider I'm pretty sure.

[mwb1100]

If the cloud drive is mapped (or can be accessed via a UNC path) you can try using robocopy which has some options that might be useful:

  - /Z :: copy files in restartable mode.
  - /IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.
  - /R:n :: number of Retries on failed copies: default 1 million.
  - /W:n :: Wait time between retries: default is 30 seconds.

[Shades]

With copying through networks in the XP days I always had good experiences with TerraCopy. Copying 4Gbyte archives and Oracle dump files over the LAN took 5 to 10 minutes less with TeraCopy than with the the standard explorer copy routine from MS. More secure too, as there was a function to verify if the hash of the copy matched the hash of the original file and it would show you a report of files that were copied correctly and which failed. Repeating the copy action would only copy the files that failed.

Now I'm on a gigabit LAN and copy speeds are a magnitude higher, even with the standard copy routine from MS, so I don't bother with copy optimizer software anymore.

Anyway, there is a piece of freeware, PathSync which is in essence sync software. The reason I mention it would be that the installer is only 130KByte in size, very easy to configure even as a one-way copier, it is very fast with checking for changes between the destination and the source, you can filter which files/folders you want and do not want to be copied or synchronized.

It also keeps logs of everything it does if you enable that functionality. The whole configuration can be stored as a file and reloaded at your desire. Also, after it is installed once, you can copy the installation folder and use it as portable application.

To say it isn't actively developed would be an understatement...the last stable version is from 2007 (v0.35), the latest beta from 2010 (v0.40).

When I discovered this software many(!) moons ago, I really appreciated the simplicity this software offers for the rather advanced features it contains. It's speed is awesome and it works absolutely fine on any version of Windows 2000 and onwards. Again, free, small and portable, what more could you want ;-)

[MilesAhead]

I would take a look at both RichCopy  and using BITS the Background Intelligent Transfer Service.  I haven't used it programmatically myself.  But according to the page it can be manipulated with PowerShell.

[IainB]

Not sure whether you are saying you have tried this: If the Cloud storage is something like (say) Google Drive or Box or Wuala, then you could download their software to make a local copy of the drive, which would then sync/update the files as they changed (at either end).
Does the Cloud storage provider you are using not offer something similar? I gather the syncing with these local drive copies is able to be paused as well, if speed/bandwidth or priority were an issue.

[MerleOne]

Syncovery supports several clouds and you may limit the transfer speed, AFAIK.

[questorfla]

Happy New Year to everyone!
Thanks for all the good suggestions.
One or more of you mentioned the use of the sync provided by the Cloud Service.
Yes, they DO provide "something" that should od it.
BUT with the caveat (provided by them) that in a case like I have now, where I first need to get a local copy equal to the cloud copy, they say to use their "restore" feature.
It got kind of complicated because of various differences in naming and the fact that the only way I could TELL what was IN the cloud copy anyway was to copy it all down.
Running any kind of "dir" check or any other on the map cloud drive gave me totally incorrect answers.

I Knew that there were more than it said there was.  But the only way to Prove it was to copy every single file down to "earth" again.  Which I began doing over the Holidays.  And I have to do this one folder at a time or sometimes one subfolder at a time if they have a lot of files.  If I don't, I start getting a ton of "errors" that show up in a "window" in their application.  These "errors" are also "too late" to do anything about.  This is all due to the way they have this whole mess configured to work through a "hidden cache" folder on the server in Program Data.  Long story but.. the best (and only working) sync app I ever found was AllwaySync. 

The company is named JungleDisk and they sell Rackspace storage through their own apps.  I had to use them due to a need for being able to have the storage accessible to the user via a mapped network drive and this all started before windows 8 was released.
Windows 8.1 I had hoped would give me a better way since it is supposed to provide native support for drive letter mapping to FTP folders but if it does, I can't get it to work.  Not without using NetDrive or Webdrive

I still don't fully  understand why those apps can so easily handle the change to a drive letter while windows (even though it SAYS to click here to get easier access via mapped drive" simple wont do it.  The best you can get is a mapped network place. which won't work for the program that needs access.  It can ONLY recognize a mapped single drive letter.

If anyone knows a way around the windows limitation please let me know.  This is also partially the reason I could not use OneDrive for the same storage.  We have tons of Free OneDrive space but I cant Map to it so.. I cant use it either.
(It also originally had a lot of other limitations but I  have had a lot of material supplied by Microsoft that indices some big changes coming this month, Jan 2015. re: One drive and I still have hopes for it)