In search of ... opinions on RAID at home

[barney]

Had a discussion group earlier today - six (6) of us this time - that started out as a football watching event.  As usual, the discussion was alcoholically fueled, and the six (6) of us voiced ~eight (8 ) opinions ... I said it was alcoholically fueled, didn't I  ?.  (Usually these events are at a table at a bar, but my recent incarceration by infirmary incompetents has interrupted that normal process.)

Anyway, the discussion today was about the value of home RAID systems, whether they are worthwhile and which RAID configurations are most practical/productive.  As mentioned, there were a number of opinions, both pro and con.  So I thought I'd bring it up here, see if there's any kind of consensus. 

So, the question is whether RAID is really practical for a home system - bear in mind that all of these guys (well, one (1) lady) are coders to some [varying] degree, so some form of backup/recovery system is important - and if so, what RAID version would be most viable?  (If it helps, three (3) of us have RAID setups, three (3) of us do not.  As well, the young lady is a C/++/# coder, three (3) of us are PHP, one (1) is ASP (Web), and one (1) is an ex-CoffeeCup employee who didn't want to leave when CoffeCup moved, don't know his disciplines.)

[flamerz]

install a ssd, and forgot about raid.

you can mirror your hard disk to and usb in a couple of minutes.

 

[Carol Haynes]

Forget about RAID on home systems. Waste of time and resources.

Horrendous problems with RAID 0 if you get a bad drive so you need to include another drive to make it more robust.

All for what? About a 15% speed increase (it should be 100% speed increase but that is pie in the sky on home systems).

Flamerz is right you will notice a much bigger improvement with SSD.

[barney]

Sorry, but I just cannot buy the SSD vs. HDD argument on any level.  SSD is about speed, but there are shortfalls there, as well as with HDD.  They both are storage media, nothing more, nothing less.  That is not relevant to the question at hand.  Hey, you could build a RAID with SSDs as well as HDDs, ya know?  This is not a question of speed so much as a question of reliability and recoverability.  Well, OK, size enters into the equation as well, I suppose, as does cost.

The question has more to do with reliability than efficiency.  Cloud storage enters into the equation, but then the question of trustworthiness and connectivity arises.  I guess the basic question has to do more with convenience - is a reasonably good backup better than a recoverable RAID system (which would also include some kind of backup).  Time factor would be a relevant, as well as reliability.  What would provide the fastest recovery?  The most reliable recovery?

I know RAID in the home is a hot-button issue (spellcheck wanted to make that headbutting  ), but I can see benefits, as well as detriments.  That's why we had the discussion in the first place  .

[JavaJones]

A reasonably good backup is better than RAID because RAID only provides 1 potential advantage, and then only when it's working well: speed of "recovery". In *theory* you can quickly and "seamlessly" recover from loss of a single drive in a RAID array by simply replacing the drive that goes bad, and you don't lose any data. With most RAID solutions there is "rebuild" time during which there will be "degraded" performance, but at least your data is there. The problems with those are several.

First of all, the chances of 1 drive in your array failing go up the more drives you have. So even though adding more drives theoretically gives you more redundancy of that data, it also raises the chances that any one component in your redundant array will experience some kind of problem. Basically, it adds complexity, and that's generally not a good thing for "home" use. Managing RAID, while simpler than it used to be, also requires more technical savvy than simple backup, again it's complexity.

Second, in order to best handle a RAID drive failure, you should keep a spare drive around to swap in. This adds to cost of the solution. Cost and complexity are both factors that tend to count more negatively in a home environment than an enterprise one, and are major reasons why RAID is generally not advisable for home use, but may be perfectly useful for business use - businesses have a higher need for consistent uptime and are willing to bear the cost to maintain that. Uptime requirements in the home are generally much less significant and of lower priority.

Third, not all RAID failure is of the expected or easy to handle variety. What if it's not your drives but your RAID controller that fails? Well, if you've done it properly with hardware RAID, you need a new RAID controller which may not be cheap, probably as much or more than replacing a hard drive, and you aren't likely to be keeping one around as a spare like you would with the drive. Also, better make sure it's the exact same controller model or it might not recognize the existing RAID array.

In the end, you need a backup of the RAID anyway, and the only thing the RAID gets you that backup doesn't is theoretical speed of recovery. But that's only if you're willing to spend the money to do it right and have a spare drive around. So you have to ask yourself, is recovering my data super quickly really that important?

The other thing is the data recovery problem is potentially easily solved with a regular backup system, assuming we're talking about simple drive failure scenarios. You just setup a frequent sync to a second drive in the system (not a RAID1, although that could be done, but is generally overkill), then if your main data drive fails, you just switch over to the 2nd. Problem solved. Or, in my case, I backup to an external drive connected by USB3, I get internal-like speeds but the unit is portable, so A: it has its own power supply and may not fry even if my computer does (e.g. my PSU does in my tower), and B: if my tower does die, I can just plug my backup drive into another computer and have my data available immediately. RAID doesn't accomplish any of that.

Bottom line, RAID adds expense and complexity that is rarely justified in a home environment. I speak from some amount of experience here, I had an external RAID unit holding a ton of my data and it died on me and I had to pay a bunch of money for data recovery. RAID is not backup, and it also doesn't always accomplish the "quick recovery and graceful failure" it promises, either.

- Oshyan

[40hz]

Unless your primary criteria is zero-downtime, it's hard to justify RAID in any scenario AFAIC.

Only disciplined and intelligent backup plan will provide genuine data protection. All that RAID can ever really do for you is reduce downtime. It does nothing to improve reliability since more drives = a higher statistical likelihood of having a drive fail.

EDIT: Whoops! JJ just got in ahead of me. Read his post. It's better!

[Mark0]

I too opted for a small and fast SSD on the PC, and a bigger and slower external unit for general storage & backup (be it a NAS or a simple external drive).
The SSD get imaged & mirrored on the NAS periodically for raw backup. Projects, websites, docs, etc. that are often updated are versioned with Git, with remotes on the NAS and on the cloud.

If going for a RAID setup, maybe for a secondary big volume, I would make sure to use a solution that don't tie me to any kind of specific hardware or software, so that if problem arises I could connect the working HDs on some PC with the right tools and be able to access the data. For a local volume, I would probably use the OS RAID functionalities; for a NAS, one based around some standard Linux stack.

The specific workload should probably also be considered. For example, for some kind of high bandwidth sequential video or audio transcoding, it would be faster to have 2 separate HDs, reading from one and writing to the other,  than the same 2 drives in a RAID 0 setup.

[barney]

JavaJones, 40hz,
You echo my sentiments, and provide some cogent arguments to present to my [presumably sober at that time] cohorts.  While I can see the usages of RAID in a commercial environment, I've never seen any real benefit in even a home office, much less a private business.

However, my compatriots presented some case studies that make RAID seem viable for a home entrepreneur.  Being as inebriated as they, at the time, I was hard put to counter their arguments - that's a downside to barroom discussions, donchano (even though this was in my living room)  .

I suppose our next discussion will be about cloud backups, as opposed to local.  I'm pretty much opposed to that scenario, but it does allow for offsite storage - if you have connectivity  .

But all of you have provided me valid arguments against RAID that are logical, presentable, and convincing, next time we hold a bacchanalia.  (Don't think it'll change the RAIDer's minds, but they'll have difficulty refuting the logic)  .

[JavaJones]

Sounds good. Tell us what they say!

- Oshyan

[mouser]

JJ made one point I think is excellent and worth elaboration, and one that I would disagree with.

The excellent point is this:

In the end, you need a backup of the RAID anyway

Even if your RAID is perfect, it only protects you from a hardware failure, it doesn't protect you from: theft, fire, accidental overwriting of important files, etc.  So when you think about RAID as a "solution", remember that it's a solution to a very narrow problem, hard disk hardware failure.

Which brings me to the part of JJ's comments that I want to take issue with.

RAID only provides 1 potential advantage, and then only when it's working well: speed of "recovery"..

I don't think that's quite right.  While that is certainly a strength of raid -- what I think raid does far better than any other solution is reduce to near zero the size of data loss due to hard disk hardware failure, and allow uninterrupted processing even in the face of hard disk hardware failure.

That is not a trivial thing for some working environments.  Just compare RAID to any other backup tool (online cloud, "realtime" file synchronization, etc.), and imagine a case where you are dealing with critical data being generated/processed at a high volume.  A sudden hard drive crash will result in a non-trivial amount of irrecoverable data with an online backup solution or a file synchronization tool, because they simply do not keep up with data written to the hard drive in real time.

So it's not hard to imagine working environments where one would want to avoid downtime during a hard disk hardware fault, and would not want to lose ANY data during such faults (which after all are far more common than fire or theft).  In these scenarios there simply is no viable alternative to RAID.

A hard disk crash remains one of the more likely unpleasant events that a computer owner will encounter.. RAID is the only true preventative solution to that very real program.  Other tools only provide the means to recover from such an event with varying degrees of loss of time, effort, and data.  Though hopefully if you've set up your other tools well and use them wisely, you can minimize the loss of data and the effort required to get back up and running.


Having said all that -- If we all agree that RAID is not a complete solution, and that you need an online backup tool and a versioning backup tool, and a drive imaging tool to let you go back in time, and your work environment is such that losing a few hours of data and a few hours of your time recovering from a hard drive crash is not the end of the world, then doing without RAID doesn't seem like a big deal.

[mouser]

We forgot to add another advantage of RAID:  Zero impact on computer load.

This is not a trivial issue -- as our long threads about poor performance of the online backup tools attests.

[barney]

That is not a trivial thing for some working environments.

-mouser (November 25, 2012, 09:25 PM)

True.  And important!  But the trade-off is the cost for a local/private installation.  As well, there's another [possible] cost than the disk drives required, although that  might be considered to be minuscule.  That would be the working environment.

For me, RAID would be redundant (all puns intended  ).  However, for a couple of my compatriots, 'tis not.  Methinks the necessity of RAID is more dependent upon need than upon practicality.  'Tis situational, I thimk.

[40hz]

A hard disk crash remains one of the more likely unpleasant events that a computer owner will encounter.. RAID is the only true preventative solution to that very real program.

-mouser (November 25, 2012, 09:25 PM)

Unless the RAID controller itself is the culprit. I've been involved in several situations where inexpensive RAID controllers introduced their own data corruption problems. And these problems rendered every element in the array absolutely useless for recovery.

And because you have data spread across multiple drives, data recovery from elements in a RAID array (other than RAID-1) is extremely difficult and generally requires expert (i.e. $$$) outside assistance.

Sometimes the greater the level of protection afforded, the greater the problems created should it ever fail.

Or as a friend of mine once put it: Slay one monster and it's only a matter of time before a bigger monster take its place.

Then there's the issue of disk make and model. E/SATA drives are not created equal. And not every drive from every manufacturer will be compatible with every RAID controller. You need to check the compatibility lists they publish. Because even though most drives will format and can be initialized as part of an array, it's only a matter of time before weird problems start cropping up. I've seen that happen with MOBO and inexpensive controller cards trying to implement RAID-5 far more often than I'd like.

There's also chipset compatibility issues with some RAID controllers.

So:

First - check your host machines specs and verify what chipset you're using.

Second - select your RAID controller, making sure it's compatible with your chipset and other system specs. And if you're doing anything other than mirroring don't bother with any controller found on the motherboard. Get a separate RAID controller card with its own CPU and battery. These are sometimes called hardware or hardware-based RAID cards. And shop quality. Don't automatically buy the cheapest controller you can find - even if it is from a "name" you recognize.

Third - get the list of recommended drives from the RAID controller's website. Use only the makes and specific models recommended for your specific controller. Also read any user forum posts on the RAID controller's website. Disk manufacturers will sometimes change the specifications on a drive without notice - and not always update their SKU code to reflect it. When that happens, the only way to discover there's a compatibility problem is if you (or preferably somebody else ) experiences it first.

 

[Carol Haynes]

if you really want RAID you need to factor in the cost of a UPS too - it is all very well having a hot swap drive gone bad scenario but power spikes (brownouts or lightening) could take down the whole array.

[Stoic Joker]

Rebuilding a RAID array does degrade performance to a mildly noticeable degree...but restoring from backup - especially an image backup - completely annihilates it. Progress and performance are both exactly zero as you sit about twiddling you thumbs waiting a few hours to get on with your life. And that's only after you get back from the hard drive store (with the replacement) which will hopefully still be open at whatever ungodly hour the thing decides to go poof at.

If you live in an area that has really bad power, then you should have a UPS anyway. If you work on large and/or important projects...and really hate suprises...then you should have a UPS anyway. It's not specifically a RAID cost.

Single disk systems, still have disk controllers, and those controllers are also just as easily subject to failure. So that's not really an exclusive to RAID issue either. So if either type of controller gets just flaky enough...data corruption can occur. But then again if a stick of RAM gets just flake enough...data corruption can occur. Yet strangely nobody ever warns against (hay more sticks is more points of failure) installing too much memory...

Now granted RAID5 is a bit much for the average home user (even if I do like running it), but RAID1 mirroring is a great way to be sure your system will be on-line when you need it.

I've seen backups fail, I've seen RAID arrays fail ...(IMO it's best to do both)... RAID failures had a lower anguish rate however.

[mouser]

If you live in an area that has really bad power, then you should have a UPS anyway.

It's silly to be even be considering RAID if you don't have a UPS power backup.. UPS is first line of defense that everyone should have.

[superboyac]

Most all good points have been made.  I agree with most of the gurus here on all that.  I've had this same argument many times with people, and there are threads here of those discussions.  I'll be honest, I hate RAID, always have.  Nobody, until the people here explained it, was able to ever convince me to use RAID.  For me, it's an easy choice: I don't like to be limited in hardware models when I build something.  If you build a RAID now and it goes down in, say, 3 years...you're going to have to buy ALL new hard drives most likely.  i don't think you will have an easy time going down to Frys and picking up a drive that will replace the one that went down.  it's a very restrictive system.  And for what?  Speed?  Don't care.  Get SSD, like they say.  If you want speed and capacity of several drives?  Use drive pooling software and use whatever drives you have lying around.  probably won't be as fast as a RAID or SSD, but it probably doesn't really matter either.  That's what I plan to do whenever I get around it...take my mish mash of drives, stick them in a box, and use drive pooling software.

[barney]

Personally, I have little use for RAID here at home.  I can see its benefit in a commercial IT environment.  And it has benefited a couple of the guys who were in the discussion, mostly because they were too damned lazy to perform proper backups  .  But I do have a backup strategy in place, so a major drive malfunction would be a time inconvenience, but little more.  Now UPSes make a lot more sense to me - have three (3) small ones, not much bigger than a surge suppressor outlet box, in my computer room  , mostly for brown power reasons, but they'll give me about ten (10) minutes to properly shut down systems in the event of an outage.  Also have a Synology NAS that can be configured as RAID, but I'd have to get a much bigger UPS to drive that in a RAID configuration.

The distaff member of that discussion group agrees - vehemently  - with me, as does one (1) other guy.  Maybe there's enough in this thread to dissuade at least one (1) other guy, but two (2) of 'em, I suspect, will not be swayed.  Like unto a religious or political discussion ... too many, "My mind is made up, don't confuse me with facts," attitudes  .  But, thanks for all the data, even if it doesn't do anything more than reaffirm my personal concepts.  We'll see.

[40hz]

I can almost understand RAID-1 (mirror) for personal use. A mirrored drive combined with a disciplined backup/sync strategy is a terrific combination for people involved in creative work where it would be hard or impossible to get back something in all its original glory if you lost it. But I wouldn't bother with any other level of RAID for personal machines at this stage of the game.

[Carol Haynes]

Mirroring is fine (if that is what you want) but I can't really see the point of the expense if you have a hardware plane.

Not sure if it is still an issue but the other big problem I encountered with domestic RAID (based on motherboard chips) was that when the motherboard died all data was unrecoverable because the formatting and RAID specifics in a striped (and I guess mall other layouts except mirrored) is defining by the onboard chip you have and the drivers. Unless things have changed radically over the years there was no standard way of formatting RAID drives and so you can't simply shift drives from one box to another if the mobo needs replacing. This means regular full backups are even more important where striped systems are used.

[f0dder]

I can almost understand RAID-1 (mirror) for personal use. A mirrored drive combined with a disciplined backup/sync strategy is a terrific combination for people involved in creative work where it would be hard or impossible to get back something in all its original glory if you lost it. But I wouldn't bother with any other level of RAID for personal machines at this stage of the game.

-40hz (November 26, 2012, 08:38 PM)

Amen.

And mirroring is simple enough that you don't really need a hardware controller for it - just go with a software OS implementation. I wouldn't touch any other other RAID levels with a 10-foot pole... more complex needs? some proper resilient clustered filesystem.

[JavaJones]

Just compare RAID to any other backup tool (online cloud, "realtime" file synchronization, etc.), and imagine a case where you are dealing with critical data being generated/processed at a high volume.

Which is exactly what makes it suitable and potentially necessary for enterprise environments and *not* for home users. How many home users do you know of that fit that criteria, "data being generated/processed at high volume"? I am one of the most demanding computer users I know and even I don't think RAID is worthwhile on my system, and I spent more than $3000 for it, so easily could have afforded it. If by "data" you're talking about lots of small files (e.g. you're a coder), then I'd still advocate a software solution in that case, because you can use a realtime local versioning system (*not* a DVCS), which accomplishes the same goal *and* improves your work by providing back versions.

Regarding backup "performance", yes CrashPlan uses a lot of memory on my system, but then it's also doing a lot more than a RAID solution would be (encryption, non-local backup, deduplication). If I turn off some of those advanced features, it's reasonable to think memory use will come down. Certainly there are lighter-weight backup (or, perhaps better yet, sync) solutions that are a more direct comparative to RAID, and while yes they inevitably have a greater performance impact than RAID, in practice it can and should be minimal. Even CrashPlan doesn't use much CPU at all, despite its high memory use.

Besides, would you not agree that RAID is *not* backup, and you'll need to be running backup software *anyway*?

Sometimes the greater the level of protection afforded, the greater the problems created should it ever fail.

Or as a friend of mine once put it: Slay one monster and it's only a matter of time before a bigger monster take its place.

(and other stuff 40hz said)
Yes, exactly. As I said, it's adding complexity, which I think most of us can agree is generally a bad word for the home user. Sure if things are working as expected, it provides benefit, but the moment something goes wrong, even the "planned for" disk failure, it starts to diverge significantly from the simplicity of the average data restore scenario. I suppose being able to replace (install) a failed hard drive should be a prerequisite for running a RAID, at the very least. But this is not necessarily as dead-simple as the average computer hardware jockey might think. If your RAID is not external, then you'd better hope you have your drives well labelled internally, because of course they're all identical. Sure, you can try to match the BIOS or Windows-recognized SATA port with the failing drive, but it's not necessarily trivial. And that can become an issue even in the "expected" failure scenario, nevermind the loss of the controller as others mentioned, or - god forbid - multi-drive corruption.

Rebuilding a RAID array does degrade performance to a mildly noticeable degree...but restoring from backup - especially an image backup - completely annihilates it. Progress and performance are both exactly zero as you sit about twiddling you thumbs waiting a few hours to get on with your life. And that's only after you get back from the hard drive store (with the replacement) which will hopefully still be open at whatever ungodly hour the thing decides to go poof at.

Unless you keep a spare drive around in the RAID scenario, you'll run in degraded mode until you replace the drive, which is riskier. You could keep a spare drive around for recovery in both scenarios. Also, RAID on the boot volume? Another complication. I was sort of assuming we're RAIDing our critical data store, and thus "full system image" backup isn't necessary. Use a simple sync "backup", your backup drive is then a 1:1 copy of your data, and you can just flip it over to primary if your main data store disk fails. In other words, the issues you point out - if they are even really issues for the home user - can be mostly dealt with using simple software and configuration strategies. That being said, I would still contend that downtime concerns of that significance are really fairly exclusive to enterprise use. After all, what home user can't just go out for a movie while their backup restores?

So, basically, RAID1 for supplementary backup purposes only, and to aid speed of recovery, *if* it's worth the cost and potential hassle to you. But frankly I just feel like it's a slippery slope to ever recommend RAID to any "home" user. The people barney is talking about sound more like IT professionals and potentially have the knowledge to deal with anything that would go wrong, so it's a lot more reasonable for them to make the (informed) choice to use it at home.

- Oshyan

[Stoic Joker]

But frankly I just feel like it's a slippery slope to ever recommend RAID to any "home" user.

-JavaJones (November 27, 2012, 11:51 AM)

Oh indeed, it would be insane to expect the average home user to wade through things of that nature. Most of them don't even bother to repair the machine in the first place. And why should they? There is nothing of any real value on it, and it was a base model $300 - $600 machine to start with ... Which is probably 2-3 years old, and is requiring a $200+ repair. Just toss the thing like any other blender/toaster/can opener appliance and get another. This is a big part of why our company does next to nothing with residential clients. Let the Geek Squad have fun contending with that mess.

The people barney is talking about sound more like IT professionals and potentially have the knowledge to deal with anything that would go wrong, so it's a lot more reasonable for them to make the (informed) choice to use it at home.

-JavaJones (November 27, 2012, 11:51 AM)

Yes, this is the class of user to which I was referring. Folks that frequently come home with important projects, and real deadlines, that can't suffer the loss of x hours to get things back up and running.

If I get a call at 2am I have two choices. Access it remotely (which requires that my comp be at least limping along well enough to fire an RDP session), or... Get my ass in the truck and head over there (there being x number of not fun miles away). I really hate option 2.

[barney]

The people barney is talking about sound more like IT professionals and potentially have the knowledge to deal with anything that would go wrong, so it's a lot more reasonable for them to make the (informed) choice to use it at home.

-JavaJones (November 27, 2012, 11:51 AM)

Yes, this is the class of user to which I was referring. Folks that frequently come home with important projects, and real deadlines, that can't suffer the loss of x hours to get things back up and running.

-Stoic Joker (November 27, 2012, 12:44 PM)

Well, two (2) of us - the ex-CoffeeCup guy and me - are retired, but have some history in software and hardware support - nearly half a century between us  .  The distaff member makes her living writing/supporting software for some businesses around Texas, so we could call her a professional, I guess.  Of the other three (3), one (1) owns or runs a repair shop for SMB machines, one (1) runs the business office for a local contractor, and one (1) has an SMB consulting firm.

So, probably not IT pros, but more knowledgeable than most when it comes to hardware/software.  And all of us have fairly esoteric home configurations, so we're no strangers to the hardware vagaries that can drive folk nuts  .

My question here was more whether there is a real - as opposed to perceived - benefit to RAID.  Personally, I don't see it.  Seems to me to be more work than it's worth.  But I've been retired for nearly ten(1) years, so I'm not as conversant with the latest technologies as perhaps I should be.  Judging by the responses here, this particular technology doesn't seem to have changed   , so my antiquated ideas are prolly still viable  .

[Shades]

My 8 year old Asus MoBo died some two months ago. It was an AMD single core linux PC (Ubuntu 10 Server) hosting 6 drives (5 SATA/1 IDE) and a software based RAID controller (RAID 6) that came with the OS.

So I bought a new modern Asus MoBo with an AMD multi-core, dreading about how much time would be lost with reconfiguring/rebuilding. Well, whining about it doesn't help so I replaced it inside the case, connecting all the drives and turned the PC on to check for damages.

Guess what, it spun up as if nothing had happened to it. I'm with f0dder, if you have to have RAID, go for software based ones, preferably ones that come with the OS.