Topic: Please help superboyac build a server (2013 edition). (Read 111,106 times)
And now they have plastic cases without the sharp edges... so what are you going to do? The reason that I liked metal cases is because it seemed that every time I cut myself, things would start working. It just likes blood, you know? Like voodoo.
Or... is that a sign of psychosis?
bofh2.jpg
(30.7 KB, 321x318 - viewed 1465 times.)
not-enough-coffee.png
(25.0 KB, 600x207 - viewed 852 times.)
^ +1You say that like there's another way...
So you are one from the (old) school of blood and magick as well...
Nope. The blood and the pain (and the risk of a tetanus infection) jogs some little grey cells and that one obvious beginner thing you thought you checked 200 tries earlier (but actually didn't) turns out to be the thing that needed doing.I'm certainly not going to deny that's it some of the time. We all know I'd be lying.
THE BOFH STILL RULEs!
THE BOFH STILL RULEs!
Hail to the Bastard!
Most of what you've read about using SAS expanders with SATA drives is just plain old fashioned FUD - stuff posted by people repeating things they didn't really understand or stuff promulgated by one particular system vendor with an agenda.I really am trying not to have to go to SAS drives, because that would just cost me quite a bit (>$3000). I want to use the SATA drives I already have. If using SATA drives with SAS is not as big a deal as I'm reading about, maybe the best way to do it is:
There is no significant issue. There was a bug in Opensolaris-derived systems that got (incorrectly) blamed on the combination of SAS expander with SATA drives because systems in that configuration could be shown to exercise the bug - but it was clearly and unabiguously a bug in the OpenSolaris code and NOT a side-effect of using SAS expanders with SATA drives. The bug existed in general and all ZFS-based systems were actually at some risk from it whether they used SAS expanders or not. It was fixed in Solaris 11 and separately on other related branches (like OpenIndiana). All of the posted FUD derives from this. It was propagated in some self-serving posts by one systems vendor who either couldn't or didn't want to bother fixing the (well published) bug in their OpenSolaris-based branch.
But if it makes you uncomfortable using then then don't use them.
Besides, as apnar posted above, serving the typical SOHO max of 24 drives using 3 8-port HBAs is almost always cheaper and faster than 1 HBA + expander. But if you need to support more drives or if you don't have the PCIe lanes to spare for the extra HBAs then expanders work great too.
Hi electricd7,
I'm actually going to be bolting pretty much that exact motherboard-processor-memory combination into my existing filer tonight!
What you need are SFF-8087 cables. They come in 2 different "flavors":
Forward Breakout = crossover = SFF-8087 (Controller card) to 4x SAS/SATA ports on DRIVES
Reverse Breakout = 1:1 = SATA/SAS Ports (on Motherboard or controller) to SFF-8087 backplane
You will need "forward" cables available here:
http://www.monoprice.com/products/subdepartment.asp?c_id=102&cp_id=10254&cs_id=1025406
I use a Fractal Designs R3 case and bought the .75M ones which turned out to be just about the perfect length to go from the card, down to the case bottom & back up to the 8 drive sleds.
While you are ordering the 8087 cables you might as well get some Molex-to-SATA converters as well:
http://www.monoprice.com/products/subdepartment.asp?c_id=102&cp_id=10226&cs_id=1022604
Use these so you only need to get 2 Molex power connectors to the drives instead of 6 proper SATA power connectors, makes for a real clean install.
-Will
Please recognize that your Windows hardware knowledge may provide some small insight for selecting hardware but is not equivalent to expertly choosing hardware for a FreeBSD based system. For example, ECC RAM in a desktop isn’t too useful. But for ZFS it can be the difference between saving your data and a complete loss of the zpool with no chance for recovery. Realtek NICs are common in the Windows world, but perform extremely poorly in FreeBSD.
FreeBSD has a very steep learning curve. It is not for those looking to learn it in a weekend. I was operating a nuclear reactor before I was old enough to drink alcohol, but I still spent a solid month getting familiar with FreeNAS. Everyone’s mileage will vary and your level of “comfort” with FreeNAS will be different than mine. But I haven’t lost any data and I have helped many people recover an unmountable zpool.
VDevs with single disks are known as “striped” disks. They have no redundancy.
VDevs can provide redundancy from individual hard disk failure inside the same VDev.
VDevs cannot operate outside of a zpool.
You cannot add more hard drives to a VDev once it is created.*
When a VDev can no longer provide 100% of its data using checksums or mirrors, the VDev will fail.
If any VDev in a zpool is failed, you will lose the entire zpool with no chance of partial recovery.
You can think of it simply as:
Hard drive(s) goes inside VDevs.
Vdevs go inside zpools.
Zpools store your data.
Disk failure isn’t the concern with ZFS. Vdev failure is! Keep the VDevs healthy and your data is safe.
ZIL drive performance will need to exceed the zpool performance of the expected workload to be useful. Typically an SSD is used for this application. An Enterprise class SSD or SSD based on SLC memory is recommended.
For maximum performance and reliability, you should never try to use ZFS with less than 8GB of RAM and a 64-bit system.
Intel Network cards are the NIC of choice. The drivers are well maintained and provide excellent performance(not to mention inexpensive). Other NICs have been known to perform intermittently, poorly, or not at all. Realtek NICs can perform decently as long as you have a CPU that has enough power to process all of the network traffic. (This is one thing that is VERY different between Windows and FreeBSD). Using “low power” CPUs such as Intel Atoms and AMD C-70s are NOT powerful enough to be used with Realtek and get good performance.
ZFS has very few “recovery tools” unlike many other file systems. For this reason, backups are very important. If the zpool becomes unmountable and cannot be repaired there are no easy software tools or reasonably priced recovery specialists you can use to recover your data. This is because ZFS is enterprise-class software, and no enterprise would waste their time with recovery tools or data recovery specialists. They would simply recover from a known good backup or mirror server.OK...this is the freaking tutorial I've been dying for!
Great stuff. I'm only about 75% of the way through, but I'm tired now.
Call me crazy, but just for backup, I'm considering getting a standard plug-and-play NAS box. Maybe from Qnap...a 4 or 5 bay one. If it becomes redundant later after i have my custom boxes working, i can put them in a relative's home for off-sute backup purposes, etc.
And I'd recommend taking a look at a Synology Diskstation- I'm very satisfied with mine, and in the end, it made me realize I didn't really need a server anymore.I'm going to have to intentionally ignore this post for nowCall me crazy, but just for backup, I'm considering getting a standard plug-and-play NAS box. Maybe from Qnap...a 4 or 5 bay one. If it becomes redundant later after i have my custom boxes working, i can put them in a relative's home for off-sute backup purposes, etc.
Nope... I call you coming to your senses
. (but duly noted)Rosewill does some nice affordable bare-bone external units. So does SANS Digital.From everything I've read so far, the thing that is sticking out the most for me is the hard drives. I have a bunch of consumer grade drives that i basically got on sale the past year or two. But they are not ideal for a freenas server, although most say that it works fine. Really, the best way to go is SAS drives. So based on this, I'm going to try a 4-bay smaller implementation of what i originally planned and test it a little bit before going huge and deciding to get sas drives or something.
I would definitely test what you're planning with a smaller incarnation before you committed whole hog to a big installation.
Might also want to check some of the forums (FreeNAS etc.) to see if anybody else has done something like what you're planning. Or tried and failed. Reason I'm saying is that there is a good chance there's a breaking point somewhere once you go over a certain storage capacity and start running into reliability issues with off the shelf open software.
Maybe somebody knows for sure one way or the other?
i mean, there must be a reason why the enterprise users don't do that kind of stuff.
MirrorThere are three options: mirror, parity, simple. So they are recommending mirror for my setup.
Stores two or three copies of the data across the set of physical disks.
Increases reliability, but reduces capacity. Duplication occurs with every write. A mirror space also stripes the data across multiple physical drives.
Greater data throughput than parity, and lower access latency.
Uses dirty region tracking (DRT) to track modifications to the disks in the pool. When the system resumes from an unplanned shutdown and the spaces are brought back online, DRT makes disks in the pool consistent with each other.
Requires at least two physical disks to protect from single disk failure.
Requires at least five physical disks to protect from two simultaneous disk failures.
Use for most deployments. For example, mirror spaces are suited for a general-purpose file share or a virtual hard disk (VHD) library.
Never tried RAID5 under software although I'm told by others that it's been reliable. There is a performance hit. But on a single user server (hmm...now that's an odd phrase) it shouldn't be a significant factor. However, I would still be concerned about the number of drives you try to incorporate under soft RAID. I'd worry with anything over three - and definitely not go past five drives in that scenario.
If I go with Windows Server 2012 and their Storage Spaces, here are my notes from that:
http://technet.microsoft.com/en-us/library/jj822938.aspxQuoteMirrorThere are three options: mirror, parity, simple. So they are recommending mirror for my setup.
Stores two or three copies of the data across the set of physical disks.
Increases reliability, but reduces capacity. Duplication occurs with every write. A mirror space also stripes the data across multiple physical drives.
Greater data throughput than parity, and lower access latency.
Uses dirty region tracking (DRT) to track modifications to the disks in the pool. When the system resumes from an unplanned shutdown and the spaces are brought back online, DRT makes disks in the pool consistent with each other.
Requires at least two physical disks to protect from single disk failure.
Requires at least five physical disks to protect from two simultaneous disk failures.
Use for most deployments. For example, mirror spaces are suited for a general-purpose file share or a virtual hard disk (VHD) library.
??? Mirroring for a VHD library? Library meaning storing them only, or running them from that location? Mirroring cuts spindle count in half which will severely limit the number of VM's you can comfortably run simultaneous.THis is not for the VMs and all that. This is just for the file storage. I need to use more than 3 disks, I'm not doing RAID. I can't tell if you guys are using the term RAID and drive pooling interchangeably. I have no desire for RAID of any kind, but I'm interested in drive pooling for the purposes of having storage capacities larger than the biggest disks available (4TB). SO I want 8TB drive pools or larger.
Original address: https://www.donationcoder.com/forum/index.php?topic=35603.0
