topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday March 28, 2024, 3:23 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Data integrity on longer term storage  (Read 2862 times)

justice

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 1,898
    • View Profile
    • Donate to Member
Data integrity on longer term storage
« on: December 01, 2009, 06:54 AM »
Okay I have data stored on an external harddrive not because it's a backup of data I am using but for longer term storage. (i think this is called archiving?). Now I am looking to create some par2 files and a checksum so I can keep the data integrity ok. Is this a good way to prevent problems with the data or are there better ways?

(yes at some point the drive will fail - can't really do anything about that atm)

Innuendo

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 2,266
    • View Profile
    • Donate to Member
Re: Data integrity on longer term storage
« Reply #1 on: December 01, 2009, 10:39 AM »
Here's what I'd do if I were you. First I'd RAR up the data into a multi-part archive with a 5-10% recovery record. Then I'd create a PAR2 dataset for recovery redundancy. And finally, I'd take the original files, the multi-part archive, and the PAR2 dataset & copy all of it to multiple hard drives.

For good measure I'd also burn at least two sets of the files onto optical media. It'd be very cheap additional insurance. If you used two hard drives in this scheme you'd have no less than 8 full sets of your data (an archived and an unarchived set on two sets of optical media and two hard drives).

Bonus points if you pair up each hard drive with a set of optical media & store each pair in spearate physical locations far apart. There'd have to be something of Act of God proportions to keep you from accessing your data later.

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
Re: Data integrity on longer term storage
« Reply #2 on: December 01, 2009, 03:09 PM »
And then store each complete set in a separate safety deposit box in a different city, preferably a different state. :D

Seriously though, how much data are we talking about here? If it's a lot, like I have (over 1TB), there's no way I'm burning it all to DVD. I tend to think multiple HD backup is the way to go, and heck even forget all the RAR'ing, PAR'ing, etc. which in itself will be a problem even for 100's of GB, let alone over a terabyte. Just get 3 HDs big enough to hold it all and copy the data to all 3. Put one in a safety deposit box (seriously), maybe give one to a friend. Encyrypt it if it's sensitive data.

- Oshyan