topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Tuesday March 19, 2024, 4:40 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Back up set of folders to flash drive  (Read 13867 times)

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Back up set of folders to flash drive
« on: July 14, 2012, 12:39 AM »

Perhaps not so much a request for someone to write a program, as just a query for best advice.

I have several folders that I'd like to back up to a USB flash drive.  Before bed at night I'd like to double-click on a desktop shortcut and have the assignment performed totally and flawlessly.

One way to accomplish the task would be a batch file, with copy commands for each folder.

xcopy c:\images\*.* /c /d /q /r /s /v /y  f:\images
xcopy c:\data\*.*     /c /d /s /q /r /v /y  f:\data
etc.

No muss, no fuss.

How can this process be improved upon?

For instance, specialized software that:

-- checks for changed files, and only copies those needed?
-- archives/compresses everything to save flash drive space?
-- pops up a final window/report showing that all copied successfully?

Please share your suggestions.

Nicholas Kormanik
[email protected]


db90h

  • Coding Snacks Author
  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 481
  • Software Engineer
    • View Profile
    • Bitsum - Take control of your PC
    • Read more about this member.
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #1 on: July 14, 2012, 01:21 AM »
Robocopy instead of xcopy :).. built into Windows. Lots of options for what you need and more.

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,640
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #2 on: July 14, 2012, 09:32 AM »
Robocopy won't do compression AFAIK, it would be easier to use a free sync program, eg. SyncBack Free, and just set a shortcut to run a needed profile.

rjbull

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 3,199
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #3 on: July 14, 2012, 03:25 PM »
SyncBackPro (the payware version) can run profiles on Windows shutdown/logoff.

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #4 on: July 14, 2012, 04:07 PM »
I'm not saying that I'd want compression.  Just wondering how you folks handle such small, quick back-up needs.

Investigating Robocopy, I see it's been superseded by RichCopy.  And then there's a competing XXCopy.  Those are just a few of the command-line approaches.

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,640
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #5 on: July 14, 2012, 09:05 PM »
There's also HoboCopy - not as full featured as RoboCopy, (as they put it), but it can use the Volume Shadow Service to enable copying of in-use files which RoboCopy can't.

SyncBackPro (the payware version) can run profiles on Windows shutdown/logoff.

I think you could use the free version and just create a task in the Task Scheduler, (it might even be what SyncBack SE/Pro does), to execute at logoff.

FWIW, I use SyncBack Pro for all my automated data sync/backup tasks - a set of tasks are automatically run when I plug in an external HDD.

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #6 on: July 16, 2012, 03:55 PM »

SyncBack Pro looks promising, 4wd.  I'm curious if you use the compression option for back-ups?  If so, do you go with the default "standard/normal" level?

Compression would seem to be a plus for saving room on the destination, if that's an issue.

Thanks for mentioning SyncBack.



nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #7 on: July 18, 2012, 08:31 PM »

Used SyncBack Pro to mirror c:\1\... to e:\1\... last night.  55 gigabytes.  Nearly a million files.

This morning I checked Windows properties on c:\1 and e:\1 to simply check if both folder trees were the same.  Most unfortunately it appears that ONE file didn't get mirrored over.

Presently using 4wd's TCBOO to attempt to find out which file is missing.

All other file diff programs seem to want to do careful file comparisons on source and destination.  Sheeesh.  Just tell me the file name that's different, please.



4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,640
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #8 on: July 18, 2012, 10:46 PM »
Used SyncBack Pro to mirror c:\1\... to e:\1\... last night.  55 gigabytes.  Nearly a million files.

This morning I checked Windows properties on c:\1 and e:\1 to simply check if both folder trees were the same.  Most unfortunately it appears that ONE file didn't get mirrored over.

SyncBack generates a report any time it runs, you should be able to open it by going to Task->View Log, they'll be opened in your default browser with details, eg. skipped files, errors, etc.

All other file diff programs seem to want to do careful file comparisons on source and destination.  Sheeesh.  Just tell me the file name that's different, please.

In Beyond Compare, uncheck the items below and it should only compare file names:

[ Invalid Attachment ]

I'm curious if you use the compression option for back-ups?  If so, do you go with the default "standard/normal" level?

I only use compression for backup of my email into one big file and that's on LZMA - Normal compression, trade off between speed and size.  There's not much gain size-wise for the more compression you use but it takes longer to compress it.

If you're talking about images, then you're better off with the Burrows-Wheeler algorithm but in my view it's not worth it due to images being mostly compressed already, (unless you're saving them as uncompressed in the first place).  Better to reduce the size of the image by reducing colours used or changing format to a lossy standard.

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #9 on: July 19, 2012, 12:34 AM »

I set up Beyond Compare as you suggested, to hopefully focus ONLY on file name difference.

There is no "GO" button.  How do I get the Beyond Compare process started?

By the way, I tried to find the one-file discrepancy between directory trees using TCBOO.  No luck.  After an hour or so TCBOO quit, with warning about something exceeded, and stack overflow issues.



4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,640
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #10 on: July 19, 2012, 03:08 AM »
2012-07-19_17-59-15.jpg

Compares as soon as it has two folders entered.

Or you can use the Context menu.

2012-07-19_18-07-07.jpg

2012-07-19_18-08-15.jpg

By the way, I tried to find the one-file discrepancy between directory trees using TCBOO.  No luck.  After an hour or so TCBOO quit, with warning about something exceeded, and stack overflow issues.

Must be a string size limit, there's not much I can do about that as long as you want the full path in the output unless I go back to using arrays and that will be painfully slow with one million entries to sort through.

Did you check the SyncBack log ?

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #11 on: July 19, 2012, 03:22 AM »

I ran SyncBack again.  It reported the two folder trees as identical.  "Success."

Just finished using UltraCompare.  It found the culprit.  A hidden file.  One automatically created by an image viewer, containing a "thumbnail database."  Good grief.

SyncBack settings apparently overlooked hidden files in the mirror process.


4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,640
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #12 on: July 19, 2012, 05:10 AM »
SyncBack settings apparently overlooked hidden files in the mirror process.

Check the Filters - there are default files/folders that SyncBack doesn't copy because it's ephemeral data, thumbs.db is one of them.

2012-07-19_20-08-36.jpg

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #13 on: July 19, 2012, 02:33 PM »

Simple as that.  Thank you!



db90h

  • Coding Snacks Author
  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 481
  • Software Engineer
    • View Profile
    • Bitsum - Take control of your PC
    • Read more about this member.
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #14 on: July 20, 2012, 02:44 PM »
I'm glad you found a solution. I still prefer Robocopy. More options than you can shake a stick at.

You *can* compress files with it (using the target FS supports it, e.g. NTFS). However, it is individually, by changing attributes at copy time, which it supports. This does hurt the compression ratio since they aren't first TAR'd (to use a nix term) together before compression. Still, it works very well and is so powerful and easy. That's why I personally like and use it. Also, it is an official Microsoft product, and free.

Other solutions can certainly be easier. Even Windows Backup in Vista and above is pretty darn good these days, but I still don't trust it as much as I do a plain old mirror of my files.
« Last Edit: July 21, 2012, 02:38 AM by db90h »

nkormanik

  • Participant
  • Joined in 2010
  • *
  • Posts: 552
    • View Profile
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #15 on: July 20, 2012, 03:09 PM »

Seems the bottom line -- whatever the tool -- is to have a backup.

Sure helps me to sleep better at night.

On my radar screen are those "network drives."  Would be great to have a big one, with built in "dlna," so the TV can also communicate with it.



db90h

  • Coding Snacks Author
  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 481
  • Software Engineer
    • View Profile
    • Bitsum - Take control of your PC
    • Read more about this member.
    • Donate to Member
Re: Back up set of folders to flash drive
« Reply #16 on: July 21, 2012, 02:39 AM »
In addition though - be SURE to test to make sure you can actually get to ALL the data in your backups. Don't let security trump accessibility for backups - unless you need that security, of course.