Author Topic: Building a home server. Please help, DC! (Read 156,797 times)

lotusrootstarch · « **Reply #100 on:** September 01, 2011, 12:28 PM »

HAHAHA. Indeed. But there is no commercial grade NAS, only SAN.

Using NAS for a serious business is like asking for trouble.

tomos · « **Reply #101 on:** September 01, 2011, 12:49 PM »

as someone who has absolutely no clue, I'm getting mixed messages reading this thread -

dont go the expensive server route
do use NAS
using NAS for serious business is like asking for trouble

if he has the money why shouldnt he get a business solution if it's more dependable?
or does it cost an arm and a leg more?

lotusrootstarch · « **Reply #102 on:** September 01, 2011, 12:56 PM »

Quote

does it cost an arm and a leg more

Definitely.

I believe all the above discussion was about just setting up a home theater, not trying to start up a baby YouTube-alike business. ROFL.

superboyac · « **Reply #103 on:** September 01, 2011, 04:09 PM »

Quote from: tomos on September 01, 2011, 12:49 PM

as someone who has absolutely no clue, I'm getting mixed messages reading this thread -

dont go the expensive server route
do use NAS
using NAS for serious business is like asking for trouble

if he has the money why shouldnt he get a business solution if it's more dependable?
or does it cost an arm and a leg more?

Thanks tomos. That's how I feel also. Oshyan is right also, I am overkilling this. But I know that, this is how I work: I overkill to the extreme, and then start paring away. It's how I learn things.

So here is where I'm at now:
I'm probably NOT going to get a server, and just a NAS.

I would prefer to get commercial quality, as I too don't like consumer quality. But if it's not available or if it's like twice as expensive, then it's not worth it. But I do like commercial quality in an eccentric way.

I still may buy server, but I'm letting go of that. It sounds like a NAS connected to my router with my desktop managing whatever I have to do with it is the solution.

And why is using a NAS for serious business trouble? That is such a vague statement, it makes no sense to me. What does "serious business" mean? It's just files and folders to me, and moving them around, renaming them, copying them...what makes something more serious than another thing?

Stoic Joker · « **Reply #104 on:** September 01, 2011, 04:40 PM »

Quote from: superboyac on September 01, 2011, 04:09 PM

...what makes something more serious than another thing?

The desire to still have it available after something goes wrong.

95% of the businesses that lose their data go out of business within 3 years. <-That was critical information going poof!)

Can you redefine the actual requirements for the project for us? I keep thinking you are wanting to get like 13+TB of data stored in a (long term) stable environment. But I could be off in the weeds a bit.

superboyac · « **Reply #105 on:** September 01, 2011, 05:02 PM »

Quote from: Stoic Joker on September 01, 2011, 04:40 PM

Quote from: superboyac on September 01, 2011, 04:09 PM
...what makes something more serious than another thing?

The desire to still have it available after something goes wrong.

95% of the businesses that lose their data go out of business within 3 years. <-That was critical information going poof!)

Can you redefine the actual requirements for the project for us? I keep thinking you are wanting to get like 13+TB of data stored in a (long term) stable environment. But I could be off in the weeds a bit.

Sure, I'll redefine it. I think you are still correct: I basically want to store anywhere from 4-10 TB of data is a long-term, stable, etc. environment. The reason for the huge range is because I don't know how quickly it can expand. Right now, I have about 4TB of stuff, a lot of it on external drives and discs. I want to centralize everything. I don't know why the amount of data bothers people. The knee-jerk reaction is always "Why do you need so much data?? What's the matter with you??" and I really don't get why it's such a crazy thing. First, hard drives are super cheap: 10 TB of hard drives is like $500, so what? Is that really that expensive and crazy?

The other thing that's a headache for me right now is backing stuff up. i want triple redundancy, which to me just means more hard drives. No big deal. I ain't losing my shit if I can help it. I've lost it once before, and I never want it to happen again unless there's an Act of God.

I just want a box that will store tons of data and will be expandable if I need to add drives. That's my only real requirement. Everything else will be dictated by how nice it is and how expensive it is. I like nice things and will pay for them...to a point. bang for the buck is what I do.

This is not the end of it. My master plan is to really tie down all of my file/folder access needs. I've been using Dropbox which is nice, but the 2GB limit is now an issue. I don't like cloud services, and I have have way too much data for cheap cloud services. I'm against paying monthly bills for anything if I can help it. So I will go to great lengths to create my own dropbox, and I've figured out a way to do it and it's awesome. I can't wait to try it out. You guys will think it's crazy, but believe me, it's awesome. It's a bit of an expensive project for something people would consider a luxury or a hobby, but it's important to me.

Lashiec · « **Reply #106 on:** September 01, 2011, 07:19 PM »

Quote from: superboyac on September 01, 2011, 05:02 PM

First, hard drives are super cheap: 10 TB of hard drives is like $500, so what? Is that really that expensive and crazy?

Of course. When you can buy a complete computer for that money, most people are going to flip over at the prospect of blowing away $500 in a bunch of hard drives for personal use. Not everyone can afford that luxury.

skwire · « **Reply #107 on:** September 01, 2011, 07:31 PM »

FWIW, I don't think 10+ TB of data space is over-the-top. I currently have 12TB of space in my Windows Home Server box.

Stoic Joker · « **Reply #108 on:** September 01, 2011, 08:33 PM »

Quote from: Lashiec on September 01, 2011, 07:19 PM

Quote from: superboyac on September 01, 2011, 05:02 PM
First, hard drives are super cheap: 10 TB of hard drives is like $500, so what? Is that really that expensive and crazy?

Of course. When you can buy a complete computer for that money, most people are going to flip over at the prospect of blowing away $500 in a bunch of hard drives for personal use. Not everyone can afford that luxury.

It ain't luxury, if you really need it.

Wasn't that long (4-ish years) ago I paid close to (if not a bit over) that for the 3 320GB drives that are (still) in my home machine now.

I've never seen a $500 computer that didn't make me wince (no that's a waste of money).

40hz · « **Reply #109 on:** September 01, 2011, 08:56 PM »

Quote from: lotusrootstarch on September 01, 2011, 12:28 PM

there is no commercial grade NAS

Hmm...

There's a huge market for something bigger than a file server that doesn't entail the complexity and expense of implementing a SAN solution. That's where NAS really shines. Several of my corporate clients already use NAS appliances. And several others plan on getting one.

So...perhaps somebody better inform NetApp, HP, Hitachi, and a host of other manufacturers, that there's no such thing as what they're selling?

JavaJones · « **Reply #110 on:** September 01, 2011, 09:11 PM »

Yeah, I'm not in the "your data needs are crazy" camp either. I've got, er... 20+TB at home. So yeah.

If you want "triple redundancy" and you *don't* need any of that to be off-site (or you're willing to do some schlepping on a regular basis to achieve off-site security), then I think your best bet is to get the largest NAS you can, buy 3 of them, use one locally to backup the other, then backup to a 3rd one on a weekly or monthly basis and keep it off-site the majority of the time.

NAS setups are simple, if you want future expansion capacity just get one with more drive bays. That Synology with 12 bays is pretty extreme, by the time you fill it up you might even want to just fully replace your NAS hardware anyway, but it's not completely unreasonable to get it. Personally I'd probably go for the DS1511+ with 5 bays and get 5x3TB drives = 15GB storage or ~11TB in RAID5. It's less than half the price of the DS2411+ as well which will help the pocket book if you want to buy 3.

Also, just because some people believe NAS is not acceptable for "enterprise use" doesn't mean A: everyone thinks that B: that "enterprise use" applies to you. Plenty of businesses (where do you draw the line between small business and "enterprise"?) use NAS products and there are business-oriented systems that have reasonable reliability, configurability, etc. The Synology units are among them.

Not to mention that with triple redundancy, well, no high-end enterprise solution is going to give you *more* reliability in a single box for less money than it would cost to achieve true triple redundancy on lower-end hardware. In other words you can have a high-end SAN system for tens of thousands of dollars that is super-reliable with redundancy and self-corrective systems, but unless you plan to buy 2 or 3 and keep one off-site, your house burning down can still kill it. With cheaper hardware, buy more of them and keep copies, with one off-site you're as protected as you're likely to get.

- Oshyan

f0dder · « **Reply #111 on:** September 01, 2011, 10:28 PM »

Quote from: JavaJones on September 01, 2011, 09:11 PM

Also, just because some people believe NAS is not acceptable for "enterprise use" doesn't mean A: everyone thinks that B: that "enterprise use" applies to you. Plenty of businesses (where do you draw the line between small business and "enterprise"?) use NAS products and there are business-oriented systems that have reasonable reliability, configurability, etc. The Synology units are among them.

...not to mention that some of the really heavy enterprise storage systems are actually NASes and not SANs

lotusrootstarch · « **Reply #112 on:** September 01, 2011, 10:29 PM »

Quote from: 40hz on September 01, 2011, 08:56 PM

Quote from: lotusrootstarch on September 01, 2011, 12:28 PM
there is no commercial grade NAS

Hmm...

There's a huge market for something bigger than a file server that doesn't entail the complexity and expense of implementing a SAN solution. That's where NAS really shines. Several of my corporate clients already use NAS appliances. And several others plan on getting one.

So...perhaps somebody better inform NetApp, HP, Hitachi, and a host of other manufacturers, that there's no such thing as what they're selling?

True. I'm not trying to deny the business merits of NAS appliances, I was just saying that there are a bunch of factors that prevent NAS from playing a serious role.

To list a few:

1. The NIC becomes a serious bottleneck with increased capacity. How long will it take to back up a 10TB NAS with a SMB transfer speed of ~35MB? Very cumbersome to move large chunks of data for purposes like archiving/making backups.

Think about it even in a home usage scenario, a normal BD burning session at 8x from a NAS can easily max out its link.

2. Lack of fine-grained access management even with AD integration.

Stoic Joker · « **Reply #113 on:** September 01, 2011, 11:00 PM »

The schism/stepping off point is (or seems to be) cost. For the cost of 10TB of BestBuy class NAS boxes one could easily just get a refurbished commercial server that will always have parts available, is designed to take 100+ times the beating you'll ever give it, and it has a proper true hardware RAID controller ... With a year warranty ... For roughly the same price.

40hz · « **Reply #114 on:** September 01, 2011, 11:29 PM »

Quote from: Stoic Joker on September 01, 2011, 11:00 PM

The schism/stepping off point is (or seems to be) cost. For the cost of 10TB of BestBuy class NAS boxes one could easily just get a refurbished commercial server that will always have parts available, is designed to take 100+ times the beating you'll ever give it, and it has a proper true hardware RAID controller ... With a year warranty ... For roughly the same price.

Amen. Exactly right!

40hz · « **Reply #115 on:** September 01, 2011, 11:53 PM »

Quote from: lotusrootstarch on September 01, 2011, 10:29 PM

Quote from: 40hz on September 01, 2011, 08:56 PM
Quote from: lotusrootstarch on September 01, 2011, 12:28 PM
there is no commercial grade NAS

Hmm...

There's a huge market for something bigger than a file server that doesn't entail the complexity and expense of implementing a SAN solution. That's where NAS really shines. Several of my corporate clients already use NAS appliances. And several others plan on getting one.

So...perhaps somebody better inform NetApp, HP, Hitachi, and a host of other manufacturers, that there's no such thing as what they're selling?

True. I'm not trying to deny the business merits of NAS appliances, I was just saying that there are a bunch of factors that prevent NAS from playing a serious role.

To list a few:

1. The NIC becomes a serious bottleneck with increased capacity. How long will it take to back up a 10TB NAS with a SMB transfer speed of ~35MB? Very cumbersome to move large chunks of data for purposes like archiving/making backups.

Think about it even in a home usage scenario, a normal BD burning session at 8x from a NAS can easily max out its link.

2. Lack of fine-grained access management even with AD integration.

I haven't found that to be the case in many of the "commercial grade" products. Most support CIFS, NFS, NCP, HTTP and FTP protocols. And virtually all have multiple 1GB NICS provisioned for failover, load-balancing, and teaming. So there's no dearth of usable bandwidth there. And many either currently have, or will soon have, FCT -so they're also SAN ready should you ultimately need to take you NAS box in that direction. Same goes for access granularity. The management software and OS is top-notch.

But this is commercial grade ($5k range and up) we're talking about right? I'm not talking about a SnapServer or something similar.

lotusrootstarch · « **Reply #116 on:** September 02, 2011, 12:06 AM »

Quote from: Stoic Joker on September 01, 2011, 11:00 PM

The schism/stepping off point is (or seems to be) cost. For the cost of 10TB of BestBuy class NAS boxes one could easily just get a refurbished commercial server that will always have parts available, is designed to take 100+ times the beating you'll ever give it, and it has a proper true hardware RAID controller ... With a year warranty ... For roughly the same price.

The running cost, the noise, the heat, the storage space... all for just 10TB in a home storage context? Hardly worth it IMHO.

I'm going OT a bit to look at this from another perspective, do we really need the "redundancy and high availability" for this kind of usage? I used to deploy RAID 10, get my AV content carefully ripped, well organized, double-checked for corruption... only to have the majority of content replaced in less than a year due to newer releases which entail better content, higher definition, 2D->3D, etc.

Take Avatar (2009) for example (I assume most of you have watched it by now), I had over 150GB data turnover on this particular title alone.

1. Avatar BD 1080p - Ripped, organized, verified, gave away and removed from storage.
2. Avatar BD 3D 1080p - Ripped, organized, verified, gave away and removed from storage.
3. Avatar Extended Collector's Edition 1080p - Ripped, organized, verified, gave away and removed from storage.
4. Avatar Extended Collector's Edition 3D 1080p - Ripped, organized, verified, currently in use, hopefully it'll last another year or so before they re-master it to go beyond 1080p.

Same theory applies to audio as well, at least in the genres that I listen to (DTS Master HD soundtracks and classics).

Thus there's not much reason to spend so much money and effort to have HA storage in the home media context.

The content is disposable and is likely to be phased out before you have a clue.

Stoic Joker · « **Reply #117 on:** September 02, 2011, 12:43 AM »

Quote from: lotusrootstarch on September 02, 2011, 12:06 AM

Quote from: Stoic Joker on September 01, 2011, 11:00 PM
The schism/stepping off point is (or seems to be) cost. For the cost of 10TB of BestBuy class NAS boxes one could easily just get a refurbished commercial server that will always have parts available, is designed to take 100+ times the beating you'll ever give it, and it has a proper true hardware RAID controller ... With a year warranty ... For roughly the same price.

The running cost, the noise, the heat, the storage space... all for just 10TB in a home storage context? Hardly worth it IMHO.

You can't possibly be serious. Servers went green to ya know?? running cost is dependent on load. no load. no cost. Sure it draws a bit at idle but without a monitor... less that any other desktop I'd wager.

Noise? Total bullshit. Anything from a PowerEdge 1800 or newer has fan speed controls. Sure under peak load they sound like a vacuum cleaner, but at idle - where they'll be spending all their time - they're whisper quiet.

Heat? Fans say there really ain't much to speak of (See above), especially if you spec it with its actual usage in mind. No 2nd Xeon, no extry 20GB of RAM, etc. etc.

Oh, and it up to the individual to decide if their data is worth keeping ... There are whole TV channels dedicated to movies from the friggin 20s. So obviously somebody had to be hanging on to that shit for some time now... Huh??

lotusrootstarch · « **Reply #118 on:** September 02, 2011, 01:01 AM »

Quote from: Stoic Joker on September 02, 2011, 12:43 AM

Noise? Total bullshit. Anything from a PowerEdge 1800 or newer has fan speed controls. Sure under peak load they sound like a vacuum cleaner, but at idle - where they'll be spending all their time - they're whisper quiet.

Well I cannot comment on the PowerEdge 1800 since it looks like it's barely entry level and you'd probably better off getting a beefy quality desktop for the price anyways. I was referring to normal servers such as DL385/R710/R810.

Quote from: Stoic Joker on September 02, 2011, 12:43 AM

Oh, and it up to the individual to decide if their data is worth keeping ... There are whole TV channels dedicated to movies from the friggin 20s. So obviously somebody had to be hanging on to that shit for some time now... Huh??

Much less applicable for young generations but granted you've got a point.

steeladept · « **Reply #119 on:** September 02, 2011, 08:35 AM »

Quote from: Stoic Joker on August 06, 2011, 02:17 AM

Hm... (Let me scamper a bit further out on the limb...)

MTBF (Mean Time Between Failure) as I understand it is the earliest statistically likely point for a given device to fail.

Now (operating completely without a net...), the odds of a coin landing heads up are 50/50. Which is to say that statistically there is a 50% chance of it coming up heads (I do believe it's safe to interchange them in that fashion ...Yes?).

The fun starts when you look at the odds of a coin coming up heads if it's flipped (oh lets say...) 3 times ... Because it is still 50/50 due to each flip being a separate event with 2 possible outcomes.

So I have a bit of trouble getting my head around the idea that the MTBF of 3 devices, is lower than the MTBF of 1 device. When they all individually have the same odds (statistically 0 until age X) of failure at any one given point in time.

Only a month behind on this particular post but whateva....

Just thought I would try to explain where you went wrong here...You are right in your example, any given time, there is a 50/50 chance of it coming up heads. The idea of MTBF is not that it will come up heads any one time, but rather what is the chance of it coming up tails any one of the 3 times. It isn't how likely it is SOMETHING is going to fail at any given time, it is the likelyhood of ANYTHING NOT failing at a given time. Another example may make it clearer: It isn't the chance of the coin coming up heads or tails any 1 time, but rather the chance it will come up heads EVERY time. MTBF is the average number of flips that will statistically guarantee heads comes up every time. With a coin it is 50/50 (as is any even distribution), but with something designed NOT to fail, it has a lifespan that can be used as a reference to statistically determine how long the product should last before it fails.

Note that statistically means you are still 50/50 on your one device/component/whatever but that all devices/components/whatever will average out to that number. When you see an MTBF of 5 years, that doesn't mean it will last 5 years, what it means is the average life of any given sample of that product will work out to about 5 years. YMMV. The real value is determining between manufacturers and/or product lines. An MTBF of 3 years for one device and 5 years for the other means you are likely to have the 5 year one significantly longer.

Stoic Joker · « **Reply #120 on:** September 02, 2011, 11:43 AM »

Hm... I'll have to mull that over for a bit, as it's early, and statistics tend to annoy me...

But thanks for taking a crack at it!

40hz · « **Reply #121 on:** September 02, 2011, 01:12 PM »

Quote from: Stoic Joker on September 02, 2011, 11:43 AM

... it's early, and statistics tend to annoy me...

Wow! Really?

I'm gonna have to introduce you to my Mom some day. She feels the same way about facts.

Facts make her angry.

Stoic Joker · « **Reply #122 on:** September 02, 2011, 02:50 PM »

I have no problem with facts. I like facts. However statistics are not facts ... They are mathematically bases guesses, or assumptions if you will.

It's a side effect of spending a great deal of my life pushing a (two wheeled) vehicle right up to and well beyond the very ragged razor edge ... of what it statistically was not capable of doing.

As Hon Solo said in the original Star Wars, "Never tell me the odds...".

I am a fan of the chaos theory, and I tend to root for the X factor.

40hz · « **Reply #123 on:** September 02, 2011, 06:14 PM »

Quote from: Stoic Joker on September 02, 2011, 02:50 PM

They are mathematically bases guesses, or assumptions if you will.

Actually, they're a lot more than that. The only problem for the general public is that statistics is such a little understood branch of mathematics that most people neither respect nor 'get' what it's about.

And considering how widely misused and misquoted statistics are, it's small wonder.

superboyac · « **Reply #124 on:** September 02, 2011, 07:15 PM »

The setup I posted above is not that expensive (~$1500). A Q-NAP or Synology NAS that can hold 10 drives would cost me $1500-2000. So the price is not really an issue. Now, I still like having commercial grade stuff, so I'm not sold on the NAS yet. What I want to add to that quote is some kind of rack that can hold at least 10 drives. Something like a Dell PowerVault:

If I add that to my setup, buy all the hard drives, and find a nice 4-foot tall cabinet for it, I think I'm set. Right?