Main Area and Open Discussion > General Software Discussion

Program to copy contents of a cloud drive to a local drive -slowly

(1/2) > >>

questorfla:
I guess that is the best way to put it.  I have a need to get the contents of our CLOUD storage onto a local system.  This is a pretty big pile of files and while I can sit here and manually do this one folder at a time, I would rather not as there are several hundred folders.  My first though was to simply run a SYNC program but I really don't need a Sync, other than maybe as a "one-way" thing.  There is nothing to Upload or sync back to.  I just need to "back-up" the Cloud.

I have used a couple like 'All-way Sync' before and maybe that is still the best way to go.  Some of the Sync programs tend to get Overly in a hurry which in this case I am not and preserving enough bandwidth to function would be a main concern. 

I need to download about 60GB of files.. Slowly!  Which in most cases is not what people are after. Using Windows, for instance, is hopelessly chaotic.  At least the GUI part.   If I try to drag and drop the whole thing at once it always crashes.  Folder by folder works but is getting tedious. 

Maybe a program that would drag and 'drop-to-copy' one folder at a time and do the next when the first is done?  Or a program that gave a way to limit the number of files it would try to copy at once?  Some way to limit the amount of computer overhead and bandwidth use so that everything copies... eventually.. and the system can run normally all the while albeit a little slower? 

The Cloud drive is mapped as a network drive and I have already begun the local folder copy of it but ran into these issues and thought I would see if anyone else has ever tried to "back-up" out of a cloud before and knew a better way.  :(  Every time I Google Cloud Backup .. Well I am sure you know what I find,  Everyone going the Other Way :)

worstje:
Try looking at file copying software like TeraCopy, TotalCopy or similar. I used these over a decade ago, so they may not be the best option anymore. You can just copy files with them as usual, but besides some retry-stuff in case of dropping connections, there's also a speed slider I'm pretty sure.

mwb1100:
If the cloud drive is mapped (or can be accessed via a UNC path) you can try using robocopy which has some options that might be useful:

  - /Z :: copy files in restartable mode.
  - /IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.
  - /R:n :: number of Retries on failed copies: default 1 million.
  - /W:n :: Wait time between retries: default is 30 seconds.

Shades:
With copying through networks in the XP days I always had good experiences with TerraCopy. Copying 4Gbyte archives and Oracle dump files over the LAN took 5 to 10 minutes less with TeraCopy than with the the standard explorer copy routine from MS. More secure too, as there was a function to verify if the hash of the copy matched the hash of the original file and it would show you a report of files that were copied correctly and which failed. Repeating the copy action would only copy the files that failed.

Now I'm on a gigabit LAN and copy speeds are a magnitude higher, even with the standard copy routine from MS, so I don't bother with copy optimizer software anymore.

Anyway, there is a piece of freeware, PathSync which is in essence sync software. The reason I mention it would be that the installer is only 130KByte in size, very easy to configure even as a one-way copier, it is very fast with checking for changes between the destination and the source, you can filter which files/folders you want and do not want to be copied or synchronized.

It also keeps logs of everything it does if you enable that functionality. The whole configuration can be stored as a file and reloaded at your desire. Also, after it is installed once, you can copy the installation folder and use it as portable application.

To say it isn't actively developed would be an understatement...the last stable version is from 2007 (v0.35), the latest beta from 2010 (v0.40).

When I discovered this software many(!) moons ago, I really appreciated the simplicity this software offers for the rather advanced features it contains. It's speed is awesome and it works absolutely fine on any version of Windows 2000 and onwards. Again, free, small and portable, what more could you want ;-)

MilesAhead:
I would take a look at both RichCopy  and using BITS the Background Intelligent Transfer Service.  I haven't used it programmatically myself.  But according to the page it can be manipulated with PowerShell.

Navigation

[0] Message Index

[#] Next page