What is your preferred server OS for home use? And Why?

[Renegade]

No vote there as I don't need a server for home use. Regular desktop OSes are just fine.

I work at home, and use servers for work, but that's another matter. For that, it all depends on the work I need to do.

[superboyac]

@SB-

A server's primary function is to provide security. It does this by controlling access to resources and data assets stored on a network. Anything else a server does after that is pure gravy.

Some suggestions in no specific order:

- If you don't need much security - and all you want to do is store and share files - a NAS solution is your best bet.

- If you want/need to do more than that (i.e. provide remote access, have user roles, offer additional services such as VMs, HTTP or SFTP) then you will want to get a 'real' server.

-For home or SOHO use, Windows Home Server is all you'll need. It's very easy to work with. And it doesn't take a pilot's license to fly it. At a street price of around $100 (so far) it's also pretty cheap for a product that has full tech support available.

-Don't even bother running print shares off a home or SOHO server. Unless you need to restrict who gets to use the printer (or account for the number of pages coming off it) just go for a printer with network capabilities. Print directly to it over your network and be done with it.

For a business with plans to grow (or just delusions of grandeur) the choices get a little more complicated.

-If you're going to maintain it yourself, it doesn't really matter what you pick. Linux or Windows - either way you've got some work and book time ahead of you.

-If you're hiring, pick whatever the most popular platform in your area is . Because that's what the talent pool you're going to hire from is likely working with - and knows best.

In my neck of the woods, it's Windows Server hands down. Out in sunny Los Angeles or Frisco Bay it's probably more likely to be an even split between BSD and Bill Gates.

-If you want to try a general Linux server solution, and it's your maiden voyage, try one of these first: Zentyal or ClearOS. They're very forgiving since they have a nice GUI to work with until earn your Techno-Wonk Beanie-copter.
 (see attachment in previous post)
(Note: Effective 10/17/1999 - having webbed-feet is no longer a requirement in order to wear a 'B-C.')

Luck!

-40hz (January 04, 2011, 07:02 PM)

nice!  This goes in my notebook.
Based on this, I know what I will do.  I'm going to start with Windows Home Server.  If my needs grow beyond that, i will deal with it then.  5 years ago, I would have gone with the most hardcore solution, probably Windows Server in this case.  But I'm different now, I've learned my lessons.  Very glad to hear, and I know most people here have said very positive things about WHS.

[mwang]

I don't have the kind of extensive experience and knowledge demonstrated by others here, but I've been using a Linux server for several years (from Redhat to Mandrake to Fedora to Ubuntu currently). I have never tried Win Server so I can't compare, but I'm very happy because:

1. it's rock solid; never crashes unless the hardware fails (see the next point).

2. it works on very old hardware, so when the server dies, I get to buy myself a new desktop, move my old desktop to my wife's desktop, and then move hers to server. This happens every few years, very good excuse for buying new hardware and trying a new distro.

With a good backup practice, switching servers is easy.

3. No need to pay big bucks for server software. I learned Postfix, Spamassassin, Dovecot (for email) and Samba (for Windows file sharing) first, then BIND (DNS) and Apache (web), and then gradually onto other stuff. All standard stuff on *nix land, with plenty of reference material and easy to get questions answered.

4. Easy to manage remotely. My server sits next to my desktop on the floor, but it has no monitor, no keyboard/mouse attached to it. All I need is Putty (except when setting up a new system of course). I edit all the config files with my favorite editor on Windows; no time (and no need) to learn EMACS or VI.

[MrCrispy]

Server 2008 R2 for me. I have WHS running on my file sharing box but with DE being dropped and newer 4k sector drives not being supported, I've decided to move away from it. I looked at various alternatives such as FreeNas, OpenFiler, even ZFS but I simply don't know Linux well enough and most open source software runs great on Windows anyway.

I will be installing Server 2008 R2, running WHS for pc backups in a VM, and maybe a few Linux distros in VM's as well. I look forward to trying out some specialized server software as well.

[Stoic Joker]

SHIT I just spent a freaking hour typing a response and the damn thing just vaporized. Board ate it or something, hell I don't know.

Now I gotta do the whole damn thing over - Where the hell is Ctrl+S when you need it!

[Stoic Joker]

Let me just start by saying that I have seen way to many people lose thousands of dollars taking shortcuts with data storage, Okay...

What exactly is the advantage of RAID in this situation?

-JavaJones (January 04, 2011, 05:42 PM)

Redundancy. If you have one drive, and that drive blows ... I mean really blows with hard parts touching or seized (which is rather common) ... you are sunk.

Now I know you just had a bad experience with consumer grade RAID hardware but you're throwing the baby out with the bath water. RAID is not the enemy, the appliance that was handling it was...

(RAID5) Two drives blowing in perfect harmony is excruciatingly unlikely, but if you want the extra cushion there is RAID 1+0 which would require that you lose three drives simultaneously to lose anything ... now that type of configuration does requires commercial hardware, but...

If you are going to have 10+TB of data kicking around the house, it's time to get (serious) off the porch and get the type of hardware that was actually designed to store that much data in a safe and comfortable fashion. And by comfortable I mean be able to repeatedly spin-up that many drives to find the file(s) you just asked it for without frying the PSU because it got a tad dusty and couldn't quite handle the request without puffing a cloud of smoke.

Not to mention that the uber handy looking flavor-of-the-month NAS appliance isn't going to be repairable in a few years because there is no compelling reason for anybody to stock parts for it. It's an appliance ... like a toaster or a blender, they're not designed to be repaired. When they break...You throw them away. So if the proprietary RAID controller (or other parts) decide to go poof ... you ain't getting another one. But you are at the mercy of the data recovery companies, and I'd rather not think of what the bill for recovering that much data would be, would you?

Off-lease commercial hardware is cheap as hell, and being mainstream they are quite common and easy to find parts for even years later because there is a market for the parts. You get a real hardware RAID controller with built in diagnostics that actually work, and the ability to rebuild after a disk failure automatically on the fly (e.g. zero down time). Not to mention they are also insanely fast. Doing a full format on a 200GB partition takes about 17 seconds ... Try that with a NAS appliance.

Either way you are looking at a grand or so ... So why waste it on something that's unrepairable and designed to die if used as advertised. Sure the last generation commercial server won't look quite as sexy in the living room, but it'll fit just fine in a typical closet and run forever in a home environment under a scant fraction of the load it was designed for, for damn near the same price!

If you're not using AD, IIS, etc. then what's the point of using a more expensive server OS?

-JavaJones (January 04, 2011, 05:42 PM)

Who said it was more expensive?!? Off lease commercial hardware (which was already paid fore once...) will either come with or have available the necessary server OS license. It's perfectly legal and endorsed by MS. The link I provided in your other thread has a price tag of only $250 for a flat out MS certified completely legal copy of Windows Server 2003. Now how much is Window 7??

I've dealt with that company many times in the past, I'll be ordering a server from them tomorrow. They are for-real, completely legit, super fast, and the customer service there can only be compared to things I saw on TV shows from the 50s.

[JavaJones]

I still don't see a compelling argument for RAID here. You're basically just saying "pro-level RAID is more reliable". Yes, it probably is. And I guess if you buy it off-lease it's affordably inexpensive (I have no experience with doing that so I can't say). The question is whether it's significantly advantageous over basic software syncing *for the home user*.

We can't use the amount of data as a gauge of how "serious" or "important" anything is anymore. In times past only enterprises has 10+ TB of data, but these days with high resolution digital cameras, home DVRs, people ripping their own media, huge music collections, and everything else going digital, we're approaching a time where 10TB will not be unusual for the average household to have. Does that now mean *everyone* should have a RAID unit?

So again the question, what specifically are the advantages over basic syncing? It seems like the only real benefit you mentioned is speed. You could argue that realtime redundancy is good to have, and it is, but is it worth the added complexity vs. a simple sync solution? Think about what superboyac has said about the desirability of being able to just take a drive and use the data on it as-is. That's simplicity, for a home user.

And how much storage does that $1000 buy you anyway? Can you toss in 2TB drives to upgrade the storage? Does the controller support that? Are they SATA or SCSI? In other words does $1000 buy you a usable and *upgradeable* large data solution, or is it a one shot deal?

As for my own situation with the Lacie unit, it's getting a bit off-topic but your proposed solution isn't really applicable to me either. I dunno about you, but I don't equate a stand-alone unit I can just plug in to a USB/eSATA/Firewire port with a complete, separate machine with RAID. I guess I could just use that machine as a sort of NAS, but the whole point to me was to have *local data access speeds* over e.g. eSATA, rather than network-limited speeds.

- Oshyan

[Renegade]

SHIT I just spent a freaking hour typing a response and the damn thing just vaporized. Board ate it or something, hell I don't know.

Now I gotta do the whole damn thing over - Where the hell is Ctrl+S when you need it!

-Stoic Joker (January 04, 2011, 09:44 PM)

I know how aggravating that is. When doing long responses, I use Editplus, then just paste and post.

[f0dder]

SHIT I just spent a freaking hour typing a response and the damn thing just vaporized. Board ate it or something, hell I don't know.

Now I gotta do the whole damn thing over - Where the hell is Ctrl+S when you need it!

-Stoic Joker (January 04, 2011, 09:44 PM)

I know how aggravating that is. When doing long responses, I use Editplus, then just paste and post.

-Renegade (January 05, 2011, 12:27 AM)

It's all text, mates

[f0dder]

(RAID5) Two drives blowing in perfect harmony is excruciatingly unlikely

-Stoic Joker (January 04, 2011, 10:48 PM)

Yet it does happen - and when it does, it's usually when rebuilding the array, which seems to be more stressful on the disks than the simple procedure of re-duping a mirror to a blank drive. Shit's probably most likely to hit the fan if you've use drives from the same batch when building your raid5, which is a cardinal sin and all that... but with the stories I've heard, I wouldn't put my faith in Raid-5.

Limiting yourself to mirroring for RAID does mean you don't get partitions larger than a single drive, but I don't see that as a super big problem - decent filesystems can abstract that need away (NTFS junctions, unix symlinks, ZFS storage pools, ...).

IMHO the only real advantage hardware raid controllers give you is battery-backed cache. And replacements for those cards tend to be pretty expensive, don't they?

[Stoic Joker]

(RAID5) Two drives blowing in perfect harmony is excruciatingly unlikely

-Stoic Joker (January 04, 2011, 10:48 PM)

Yet it does happen - and when it does, it's usually when rebuilding the array, which seems to be more stressful on the disks than the simple procedure of re-duping a mirror to a blank drive. Shit's probably most likely to hit the fan if you've use drives from the same batch when building your raid5, which is a cardinal sin and all that... but with the stories I've heard, I wouldn't put my faith in Raid-5.

-f0dder (January 05, 2011, 05:21 AM)

I did say unlikely, not impossible, and with other configurations like RAID 0+1 you gotta blow all three at exactly the same time. Everybody has a horror story about an uncle that got blown to bits in a taco eating accident...The point is that it is highly unlikely.

Now a software RAID controller probably will torch your data if you look at it funny ... Which is why I like to avoid them.

Limiting yourself to mirroring for RAID does mean you don't get partitions larger than a single drive, but I don't see that as a super big problem - decent filesystems can abstract that need away (NTFS junctions, unix symlinks, ZFS storage pools, ...).

-f0dder (January 05, 2011, 05:21 AM)

Um... I don't recall making that distinction/recommendation, so I'm not sure what you're driving at here.

IMHO the only real advantage hardware raid controllers give you is battery-backed cache. And replacements for those cards tend to be pretty expensive, don't they?

-f0dder (January 05, 2011, 05:21 AM)

From the manufacturer? Staggering ... That's why you buy it from the other guy... The key is that the necessary part will actually be available.

The drive failure alarm is also a handy feature...especially if you have a headless server that doesn't allow you to see the first failed disk warning until it's too late.

Transplanting a RAID set is also smoother and safer with commercial hardware. When our PowerEdge 1600 started to die, I grabbed a PowerEdge 1800 for about $600 and since both used the same RAID controller (options, less is more) I was able to just drop in the second set and go. The controller pulled it in auto-magically, and Windows only needed an Import Foreign Disk reminder and we were live. Sure I could have restored the *)GB of data from tape - in a few days - But I like things simple.

[40hz]

IMO, RAID is oversold when it comes to small server deployments.

FWIW I think it's generally a big mistake to try to do RAID on the cheap. Epecially if you're doing RAID-5. I've seen too many bad things happen with inexpensive RAID controllers and consumer grade SATA drives to be very comfortable recommending them.

I believe most people would be far better off with some combination of routine backups, smart folder synchronization, and restoration disk images unless they have the money (and need) to go with "server grade" drives and controller cards.

By all means go with multiple drives inside a small server. Just don't create arrays out of them unless you have a genuine need to do so.

----------------

(RAID5) Two drives blowing in perfect harmony is excruciatingly unlikely

-Stoic Joker (January 04, 2011, 10:48 PM)

Yet it does happen - and when it does, it's usually when rebuilding the array, which seems to be more stressful on the disks than the simple procedure of re-duping a mirror to a blank drive. Shit's probably most likely to hit the fan if you've use drives from the same batch when building your raid5, which is a cardinal sin and all that... but with the stories I've heard, I wouldn't put my faith in Raid-5.

-f0dder (January 05, 2011, 05:21 AM)

+1 on that. I've seldom seen a rebuild go 100% smoothly. Usually all of the drives were purchased at the same time  - and exposed to the same environmental conditions (i.e. heat) during their service period - so the likelihood of a second drive failing during a rebuild is not exactly uncommon.

In a mission critical setting, the only thing RAID really gives you is uninterrupted uptime and the opportunity to get the "most current" backup or image off the array. If the rest of the drives in the array are over a year or two old, you're better off replacing all of them.

If I got $5 for the number of times I've replaced one RAID drive element only to have a second one go within a few weeks...well...maybe I wouldn't be wealthy, but I'd still have a tidy sum tucked  in my pocket.

IMHO the only real advantage hardware raid controllers give you is battery-backed cache.

Plus better reliability and (maybe) some speed gains along with reduced stress on the server. But on a modern server equipped with today's multicore CPUs and RAM configurations, I don't think it's all that big a benefit any more.

And replacements for those cards tend to be pretty expensive, don't they?

Less so than in the past. But they're still pretty pricey.

[app103]

One of the things I liked about Windows Server 2008, when hollow was running it, was that multiple users could be logged in at the same time, running whatever apps they needed to run, each having their own desktop.

Hollow had set up an account on his machine for me, which I accessed from my old slow WinME box over 33.6k dialup, using an RDP client. I was able to run stuff on his machine that I could never even dream about on my own box. Utilizing his broadband connection to download large PDF files and view them using his system's resources, it took me a fraction of the time it would have taken me to add the books to my ebook directory.

All while he was using his PC for whatever purposes he needed it for.

It got me thinking about how great it would be for family use, where you could give the kids an old junk PC with 9x and an account on your machine running Server 2008. Everything they would be doing would be through your machine, which you could keep the security locked down tight and easily control when they could and couldn't use it.

An old 9x box is dirt cheap to acquire (probably could even get one for free), and they are easy to maintain, especially when you don't have to worry about installing software on it, so if the kids damaged their pc in any way, resetting the OS would be quick and painless with no data loss (all their data would be on your machine). And if the hardware died, no sweat off your back to replace it with another junk PC.

[Stoic Joker]

I still don't see a compelling argument for RAID here. You're basically just saying "pro-level RAID is more reliable". Yes, it probably is. And I guess if you buy it off-lease it's affordably inexpensive (I have no experience with doing that so I can't say). The question is whether it's significantly advantageous over basic software syncing *for the home user*.

-JavaJones (January 04, 2011, 11:10 PM)

Well... with 10+TB of data ... Size Matters. Where are you going to sync it to?!? The cloud? Total recovery time = 9 weeks... *Joy* ...I ain't got that kinda time, do you? another NAS appliance? Ouch $$$.

We can't use the amount of data as a gauge of how "serious" or "important" anything is anymore. In times past only enterprises has 10+ TB of data, but these days with high resolution digital cameras, home DVRs, people ripping their own media, huge music collections, and everything else going digital, we're approaching a time where 10TB will not be unusual for the average household to have. Does that now mean *everyone* should have a RAID unit?

-JavaJones (January 04, 2011, 11:10 PM)

You gota back it up somehow, some where, and magnetic tape is too expensive for an archive that size even for most SMBs. Hell you're talking about a $10,000 device. Sure you can use one of those external drives (a stack of them actually) but do you want to walk a typical end user through an incremental restore? I don't.

So again the question, what specifically are the advantages over basic syncing?

-JavaJones (January 04, 2011, 11:10 PM)

Cost. Syncing requires either a 1 to 1 target size or you gotta decide what not to backup ... 'cause it won't fit.

It seems like the only real benefit you mentioned is speed. You could argue that realtime redundancy is good to have, and it is, but is it worth the added complexity vs. a simple sync solution?

-JavaJones (January 04, 2011, 11:10 PM)

How is it complex? You read the instructions, do it once, and then it continues working all by itself. Syncing requires that you remember to do it; or store you files in the right magic folder; or notice the error message... You aint gonna miss the alarm on a RAID controller going off ... Last time that happened here I had half the company in my office frantically pointing inside of 3 minutes. The other half of the staff...? ...Was outside thinking it was a firedrill.

Think about what superboyac has said about the desirability of being able to just take a drive and use the data on it as-is. That's simplicity, for a home user.

-JavaJones (January 04, 2011, 11:10 PM)

I did, parts of it are possible, and parts are science fiction... *Shrug*

And how much storage does that $1000 buy you anyway? Can you toss in 2TB drives to upgrade the storage? Does the controller support that? Are they SATA or SCSI? In other words does $1000 buy you a usable and *upgradeable* large data solution, or is it a one shot deal?

-JavaJones (January 04, 2011, 11:10 PM)

You can get either SATA or SCSI (SCSI typically being cheaper and smaller these days). But get the one you already have drives for, all you need are the drive trays ($10 - $20 each) and you can use the existing drives you have now. You're only limit is the number of Hot-Swap drive slots (6, 8, 10, more) there are on the machine. Yes the 2TB (SATA, I'm not sure on SCSI - probably if you can afford them, I can't) drives will work.

As for my own situation with the Lacie unit, it's getting a bit off-topic but your proposed solution isn't really applicable to me either. I dunno about you, but I don't equate a stand-alone unit I can just plug in to a USB/eSATA/Firewire port with a complete, separate machine with RAID. I guess I could just use that machine as a sort of NAS, but the whole point to me was to have *local data access speeds* over e.g. eSATA, rather than network-limited speeds.

-JavaJones (January 04, 2011, 11:10 PM)

Hm... GBit limited Network speeds. How often are you moving what size file(s)? It's not like browsing the filesystem is going to lag with the traffic of a home LAN (which everybody has these days - Even if they don't know it). How many of those Units do you have in order to be able to store the 20TB of stuff you mentioned having? They can't be that cheap.

[Eóin]

+1 on that. I've seldom seen a rebuild go 100% smoothly. Usually all of the drives were purchased at the same time

-40hz (January 05, 2011, 07:08 AM)

But the solution, surely, is just don't do that. RAID reliability assumes the probability of one drive failing is independant of the others. While that ideal can't be met in practice, you can get close to it by buying different drives from different sources and ideally different manufacturers.

[40hz]

@Eóin - you are quite correct. But I wasn't referring so much to the manufacturing batch numbers as I was to them likely all being on the same engineering revision, equally old, and equally (ab)used in whatever environment they're in.

In my experience, its environmental factors and age that have the most bearing on a drive's useful life expectancy. I haven't run into too many manufacturer's defects when it comes to server drives.

[f0dder]

Hm... GBit limited Network speeds. How often are you moving what size file(s)? It's not like browsing the filesystem is going to lag with the traffic of a home LAN (which everybody has these days - Even if they don't know it).

-Stoic Joker (January 05, 2011, 07:45 AM)

Todays' drives are going to give you ~100MB/s transfer speeds - a typical gbit LAN speed is closer to 30-40MB/s, and with a lot worse latency as well. Quite a difference if you're shuffling large media files around

[Stoic Joker]

Hm... GBit limited Network speeds. How often are you moving what size file(s)? It's not like browsing the filesystem is going to lag with the traffic of a home LAN (which everybody has these days - Even if they don't know it).

-Stoic Joker (January 05, 2011, 07:45 AM)

Todays' drives are going to give you ~100MB/s transfer speeds - a typical gbit LAN speed is closer to 30-40MB/s, and with a lot worse latency as well. Quite a difference if you're shuffling large media files around

-f0dder (January 05, 2011, 09:35 AM)

right, knew that part ... That's why I was pondering his usage/habits. I also used the word browsing for that reason. If moving a music CD the speed difference isn't a big deal. Now if you're moving a 25GB BlueRay .iso, well then yeah...Speed is a factor.

So if everything is scattered about in little data huts that would be a huge (mess) impact. But if you RDP into a server to move media files around locally there ... (hehe) ... Zoom! Which is what I do with the ton of (OS and software) ISOs I have. Granted I don't have to futz with them frequently, but when I do it never takes long.

Anyhow the question is how often does a huge media need to be moved local <-> remote? Both viewing and burning can be done over the wire. So (I am guessing here) really it's just a case of incoming organization and the occasional cleanup of stuff nobody needs/likes anymore.

[Dormouse]

I've had a NAS for many years. Set up with hardware based RAID. I've also used backup programs for many years.

Increasingly, though, I have become concerned about how much security they really bring. I've not had a failure with either (yet); at least, that I know about. And it is knowing about it that I really want to do. A backup is always a worry unless you check every time that it has everything and you can read it. I'm now much happier just syncing all my data files, and periodically copying them in their entirety to other drives/places (with copies of various stuff in places on the net). It is very easy to copy TrueCrypt volumes too. It means I can check or use either set of files in normal use; I always sync in one direction only (feeling that gives me a bit of extra protection against something going wrong), so I have to remember to save everything on to one drive only, but that is not a problem. I also image my OS/App drive regularly. I know most of them are OK because I do check from time to time, and I can always rebuild from scratch if necessary anyway.

And my NAS? Well, when I get around to it I will just copy everything off it, reformat it without RAID and use it as separate drives. I just don't see that RAID is bringing me anything I need right now. With the current price and speed of usb3/sata3 drives & SSDs, it seems best to just add them as I need them.

PS I voted for FreeNAS, because that is all I would use a home server for. If I wanted a real server, I'd go Linux. Much more familiar with Linux servers than Windows Servers.

[Stoic Joker]

In my experience, its environmental factors and age that have the most bearing on a drive's useful life expectancy.

-40hz (January 05, 2011, 08:13 AM)

Agreed, but that doesn't necessarily mean they'll be likely to fail like dominos. Our DC here is about 7 years old, two of the 3 drives in it failed 8 months apart about 2 years ago. Assuming the third should be failing shortly I ordered two drives when the second one failed ... I'm still sitting on the spair as disk three ain't even blinked yet. *Shrug*

We also have an HP Netserver LH3 (Dual PII 400s - WWwweeeeeeeeee) that was purchased back when the company first opened almost 20 years ago (It was originally a Novell box). It still has two of it's original 3 drives in it running Server 2k3 24/7 (and it once went almost 2 years without a reboot). I ordered it's replacement today (Dell PowerEdge 2800) soley because it's just too old for the latest update of the accounting software (which is all it does). I'll probably keep it running in the lab just to see which one of us dies first...

[40hz]

^What can I say? Your individual performance and mileage may vary. 

Got a Northgate 386-20 with a 80Mb drive in it that still boots up WFWG on cue everytime I hit the switch. I've had two server drives go south on me twice (different machines) in the last 6 months. Top quality units too. Both were members of RAID arrays. Both cases encountered a second member drive failure - one during the rebuild (RAID-1); the second (RAID-5) one month and 4 days after the rebuild.

Saw that exact same thing happen once the year before that.

Please underdtand I'm not trying to read too much into it. We all know drives fail all the time. 

But it has made me reassess my opinion of exactly what RAID brings to the table. And also what representations I'm making to my clients about it.

[Stoic Joker]

I hear ya... I had a client with 4 drives in a RAID5 array, one drive failed ... but nobody said anything.  I got there 3 months later after they'd been screwing with it for awhile. I managed to get it to rebuild and the box came back online. There were errors showing in the logs that another disk was failing, so I shut the box back down and told them not to run it until we got some replacements. They did not listen and flipped the thing back on the instant I left the parking lot. Three days (of constant use) later two more of the drives failed.

To the greatest of plans there is always the perfect flaw...

Whoever set up their backups did it wrong. Data Was UnRecoverable.

But other than that my experience with RAID has been positive.

[JavaJones]

Stoic, one of the things you seem to be implying above is that RAID can be a stand-in for backup (the comment about how do you backup 10TB of data). As far as I understand that's contrary to common wisdom about RAID. Working on that assumption then, even if you do have it in a RAID, you should back it up anyway, in which case if the main benefit of RAID is its redundancy, then a good sync/backup system gives you the same benefit. Hence why I don't really see the value of RAID in a low-cost, home user environment. For enterprises where they can afford to have both high-end RAID *and* a good backup system, and where high uptime is a requirement, RAID absolutely makes sense.

When I said "complexity" I wasn't necessarily referring to the end-user interaction and maintenance, more to the complexity of the *system* (and I think you significantly overestimate the complexity of a properly setup syncing system - it's essentially automatic and largely maintenance free). A RAID setup is inherently more complex in terms of hardware than a bunch of independent disks running off SATA on the motherboard (for example). The more "moving parts" and interconnectedness, the more a failure in any part of the system can affect the whole, even if redundancy is a part of the architecture (as it is in RAID). A prime example of that is the battery backup for cache operations on a RAID controller. What if the battery fails, does the user know how to replace it? (and with older off-lease equipment wouldn't it be even more likely to fail?) And if the power goes out while the battery is dead, the whole array could be corrupted. Power goes out with a stand-alone SATA drive and the worst thing that could happen is the drive that is being accessed at that moment has a bit of corruption. In 15 years of working with home and small business PCs I've not seen anything more than minor file corruption from power loss issues.

As to my data setup and speeds of access, the 10-20TB are on my media "server" and I don't usually access those remotely. I watch movies through my HDTV straight from the media machine. The only thing I will occasionally stream to other devices in the house is music, or maybe a TV episode (usually just do Netflix for that though). If I had more TVs in the house this might be more of a concern.

Anyway, what I do with the Lacie 4big is quite different. I have all my digital photos (about 60,000, around 300GB), my work files, project files, Terragen files (100GB or so), etc. on there. In truth I only have about 1.1TB of that filled up. And that's actually the most important data for me to back up. But access speed is important because I work with large image files frequently, and don't want to be pushing and pulling them over the network all the time. Imagine browsing an image catalog of 20+MB images. Flick through a few images and you'll see slowdown on the network, but with a modern, fast drive, on local access the speed is fine. I can't speak for superboyac's needs as far as backup/redundancy or external access on his proposed 20TB of data, but if he's accessing it over the network, any speed benefit of RAID will be lost anyway.

- Oshyan

[Stoic Joker]

Stoic, one of the things you seem to be implying above is that RAID can be a stand-in for backup (the comment about how do you backup 10TB of data). As far as I understand that's contrary to common wisdom about RAID.

-JavaJones (January 06, 2011, 06:23 PM)

Many of the really high-end deduplicating RAID6 storage devices claim to be able to eliminate the need for a backup. However I'm not entirely sold on that idea either. I prefer to have RAID as a first line of defense, and then have a backup for those "special moments" when shit really hits the fan.

Here's the thing, from SMBs to SOHOs down to the average Home Owner, everybody is on-a-budget. So what typically happens is enough hardware is bought to store the data being used. Some type of backup is discussed and is maybe even used a few times. Then the data grows and the space needed to store it gets bigger. Now it all no-longer fits on the backup, so the "important stuff" discussion ensues. Either that or somebody comes up with the "Let's do an incremental backup" idea, and nobody has the forethought to shoot them for saying it ... before the plan is put in place.

Note: I'm cringing right now just thinking about the number of time I walked into a new client to find they've been using the same five pieces of media to do incremental backups, for the last five years ... Chance of a successful recovery? Zero!

Both scenarios tend to end badly as backups just love to fail. And they do so at an extremely high rate. RAID on the other hand, could fail. Synchronization & backup schemes both have the same (glaring) flaw. The are simple, easy, set-it-and-forget-it systems. ...and it's that last, forget it, part that bites'em in the ass every time!

Synchronization, like mirroring (RAID1) both like to give people two identical copies of the same error. So if a disk fails in the middle of a sync... (point here being every system has a weak point)

Now for the average end user that comes trotting home with a brand new PC, a few ideas of what they can do with it, and maybe even a shiny new digital camera... Obviously a server would be a completely stupid to even suggest. (I'm picturing Goofy from the old Disney driving cartoons trotting out of a BestBuy here)

However, you and SB are not typical end users. You have an established system architected into a strategy that is so far working for you. Great go with it. SB on the other hand, is looking to start-from-scratch (so to speak), and is coming out of the gate planing on warehousing 10+TB of data. Those NAS boxes ain't cheap, as anything with more than 2-3 slots is well over $1,000. So if you looking at budgeting enough money for three of those ... You are much better off going with the off-lease commercial hardware that will give you a true scale-able hardware RAID configuration for a fraction of the money you would have spent on the consumer grade stuff.

Even if you are working out of your mother's basement (not saying you guys are), with 10+TB of data you have an enterprise class problem ... Which should be handled in much the same way regardless of where the data is and whom it belongs.

I'm hoping I covered everything as this has takes awhile for me to type.

[superboyac]

Thanks Stoic, believe me, i read every word of that.
What I need to do is sit down and visually map out my backup strategy.  Once I do that, I can post it here and everyone can debate all the weak points.  But before I do that, and I imagine it will take me a while at the rate I'm going, I do have a few main points to discuss.  Please keep in mind that I do not understand everything I am about to say in the detail I am used to understanding things before I act on it.

--I STILL do not understand how RAID is useful to me.  I'm not trying to backup an OS or anything.  Just standalone files and folders (mostly media: music, video, docs, etc.).  I still consider RAID more of a performance thing than a backup.  I get the rebuilding thing, but I don't see any advantage compared to just copying all my files/folders onto a fresh hard drive.  That, to me, is the same thing as "rebuilding" a drive.  The benefit is that the physical drive is a standalone unit, which I really desire.  I LOVE the idea of pulling a drive out at anytime and using it in any other computer.  I don't like the idea of RAID for my purposes.

--The one thing I am worried about and will probably spend the most time thinking about: how to prevent BAD backups and only have good backups.  Let me explain.  Let's say I have a hard drive, and I back it up with double-redundancy using two more hard drives and syncing (SFFS).  Now, let's say the original drive got infected with a virus that got into a bunch of my files.  Soon after, that virus will be backed up to three places.  And now I don't have an original GOOD version anywhere!  How do you prevent that?  It doesn't help if you back up with multiple redundancy if they all have the same BAD files.

So then I think, well, the backups should be staggered somewhat.  In other words, maybe one drive backs something up once a week, and the other once a month.  That's good, right?  Eh...it's ok.  I'd like something better.

Then we start thinking incremental backups or versioning.  Well, I'm not really a big fan of that either because, for one thing, the files are not standalone anymore.  You're going to need some kind of software or something to extract whatever file you want from the backup.  And again, it's not a exact copy of the files/folders...it's more like an archive of files/folders.  The other problem I have with it is I can't incremental/version everything.  Otherwise it would be too big.  So then I have to think about what's important and what's not, and I don't want to do that.

So I just don't know what the optimal way to backup things is.  I'm sure all of this has been well thought out for big companies.  I can't imagine big companies not having something rock solid and that covers all bases.  But then again, I'm never surprised anymore if that's not the case.

I just don't know what the perfect solution is.  I've read mouser's long backup document, and I even wrote a shorter one here myself.  But there are too many holes right now in my strategy, and furthermore, it's a puzzle that I haven't thought through long enough.

Cmon people, we're talking here as if it's the first time we've all ever thought about backing up.  With all the computer people here, and professional IT people, someone MUST have a really solid solution that is already in practice, no?  Am I wrong?