Got the Cabinet, now I need the guts, please help :)

Heofz

Heofz

Heofz

Soldato
Joined
20 Oct 2002
Posts
3,467
Location
London, UK
Hey guys & gals,

I've got me a 21U cabinet to host my server gear, though it's basically for networking so I can only really have 450mm depth equipment installed in it (500mm absolute tops).

What I'm aiming for is to put in a Windows Server on maybe a 4U enclosure, but have the rest of the rack space reserved for storage. I'm thinking perhaps having the server itself with several mini-SAS ports and have a number of mini-sas drive enclosures?

I would like to have the entire lot running RAID 10, and be easily expandable as I accumulate more data and thus purchase more hard drives.

Looking at 30 TB's worth of space initially (so 60 TB in drives), plus server and storage interface gear.

Not really sure where to go from here though as I've never made a purchase of this size before. Can anyone make any recommendations or have any advice?
 
I picked up a couple of Dell MD1000 disk shelves that should fit a 500mm chassis. They hold 15 drives each, work with SATA and should be fairly cheap second user.

There are few rack mount chassis that are so short. Supermicro do some mITX 1U chassis that would most likely work but you would be limited for expandability.

Throw in a something like a Intel S1200KP with an E3 Xeon (or i3 if you want to start small) and you will get dual intel lan ports and can put a SAS controller in the PCIe slot with external ports for connecting to he disk shelves.

Another interesting set of options are the recently released ASRock server boards.
E3 Haswell boards (dual lan, ECC ram compatible, KVMoIP with dedicated lan port).
E3C224D2I: 4x SATA II & 2x SATA III
E3C226D2I: 6x SATA III

Atom
C2550D4I
C2750D4I

It is generally very hard to find short rack equipment for what you want to do. The short depth cabinets are for network equipment, as you quite rightly said, and not for servers, hence not a lot of server stuff that will fit about.

RB
 
leigh_boy said:
what kind of data will you be putting on to the server?
Click to expand...
Large media files, namely compressed videos. Produced/Encoded on a different machine, and the final product uploaded to the server machine. Primarily for playback over the network as well as a big screen connected directly to the server.

RimBlock said:
I picked up a couple... ... ...
Click to expand...

Interesting, I'm trying to learn the right terms to use as I go along, but am I right in saying I'm best looking for a "Direct Attached Storage" setup?

I'll have a good scour on eBay tonight for an empty MD1000 or something similar. And you're right, so far it seems virtually impossible to find a 45-50cm depth disk enclosure!

Is there a particular SAS controller you would recommend for my setup? Anything I should be particularly aware of, i.e. paying £1000 for a card when a £100 one will do. Also I assume I shouldn't run every disk in a single RAID10 array, and instead split the logical volumes up into smaller chunks?

Does the disk shelf actually determine the RAID capabilities etc? Or does it simply act as a "shell" with power and so the RAID stuff is determined by the SAS controller card in the server?

Apologies for all the questions, trying to get as much info as possible now as I want to purchase the right stuff first time :) (Sadly the depth of cabinet was not an option due to room space limitations.
 
Last edited:
Yep, a DAS setup would be your best setup but be prepared for it to be fairly hot and possibly a little noisy. If you do pick up a MD1000, make sure you connect both power leads or the fans go in to top gear.

You will still have fun finding a case to fit for the actual server. Maybe a tower server put on a shelf may fit and be the best option. I had the HP ML110 G7 on a shelf in my rack for a while. I don't recall the depth though.

Anything LSI 2008 based is usually pretty good. LSi 9200-8e or Dell H200e are fairly good entry level controllers with external connectors. Your server can perform the raid 10 duties depending on OS. No need for an expensive SAS controller.

If you look to move up to Raid 5/6 then get a decent controller with either FBWC (Flash Backed Write Cache) or cache with a BBU (Battery Backup unit). Either will prevent corrupted data in the event of a power outage / server failure. If you have an interest then you can also take a look at Solaris or one of the free distros that have ZFS built in and use that for your data protection. It is highly resilient and free (excluding Solaris support licenses of course).

RB
 
Oops forgot about this thread! doh.

I'm about to buy a couple of MD1000 units, and just picked up an LSI MegaRAID 9280-8e on eBay for £200 :)

Reconsidering my RAID option, will probably go for RAID 6 in the end. Initially didn't want to fork out £800 or whatever for a new raid card but £200 seems to be a bargain for this card.

I guess if I'm doing RAID 6 I should buy a battery unit module to go with the 9280 for resiliency... worth it or a waste?

Also anyone know how much of a ball-ache it is to extend the size of an existing RAID 6 array to add additional drives in future? I imagine there's a lot of rebuilding that has to happen?
 
Heofz said:
I guess if I'm doing RAID 6 I should buy a battery unit module to go with the 9280 for resiliency... worth it or a waste?

Also anyone know how much of a ball-ache it is to extend the size of an existing RAID 6 array to add additional drives in future? I imagine there's a lot of rebuilding that has to happen?
Click to expand...

You'll need to look a little closer into the Battery Backed Write Cache (BBWC), You may find that RAID options using parity calculation (RAID 5 and 6, or 50, 60 etc..) require the BBWC to be available. Or if unavailable require that write caching is disabled which seriously hampers the speed of the card.

Adding disks should be straightforward, it isn't having to rebuild data using parity, it is taking existing data and leveling it across the new disks - That's not to say the process doesn't take time, but it shouldn't be as lengthy or as nerve jangling as awaiting a rebuild.
(I can't find any stats on a rebuild vs 'leveling' without finding a bajillion posts about WoW clans, it's based on past experience and gut instinct)

With the amount of disks you are suggesting you would want to be considering hot spares too, Waiting a few days for a replacement disk and hoping nothing else happens, when one fails on a Thursday night before Easter, might be more stress than you want to put yourself through.

And finally, depending on budget, proper SAS drives would be worth considering (rather than consumer SATA drives). They have improved error checking and correction, which should give more chance of advanced warning of a disk failure.
 
Cool, I definitely want to try keeping the budget down on the drives, the SAS drives are nearly twice the cost of these for the same space, and I intend on eventually owning 45 of them :D

Will look into the BBWC stuff.

Also looking at buying a 3U ATX chassis... found a few 45mm depth ones online, but was wondering: Are there some widely known ATX mobo server grade setup/configs that will serve well as general purpose with decent expansion options? Presumably a regular desktop mobo/cpu/etc will not be suitable due to power usage/heat output/etc?

Obviously the mobo will need to have the PCI-e ports for the RAID card... but beyond that and what I mentioned above, anything glaringly obvious I should be considering?
 
The 450mm/500mm length limit is tricky, I was going to suggest the Dell server outlet as an option, but you will have to investigate further on the lengths of them.

Supermicro are known for their 'homebrew' servers, so they could be worth a look. I believe RimBlock has used them fairly recently so may be able to advise further.
 
Ok wow, I totally didn't anticipate the cost of the Dell SATA caddies... £34.20 each :eek:... how is that justified? Just because they can? :confused:

If I want to buy 30 of them it's gonna cost me £1026, for empty disk trays!!
 
You will be hard pressed to find a server grade chassis with those depth limits.

The popular ML110 G7 is 457mm deep and has little spare room in the chassis. It is a shorter server chassis.

Supermicro do a range of 1U short chassis but even those are 500mm+.

You will need to factor in cable plugs on top of the chassis length.

Companies like iStar do some short chassis but they are more desktop grade. For home they should be fine.

You may be able to find an industrial chassis if you search around but it will be ugly and expensive :D.

Best bet would probably be a MicroServer G8. You can upgrade the CPU if needed, add the SAS card (check the size is ok), hook up the disk shelves and away you go. Not quite rack mount but probably the best solution given the info supplied.

I ended up getting some 9U, 1000mm units manufactured for my customers who wanted a small home lab but who didn't want a half or full rack.

As far as the disk caddies go... welcome to the server world. Just like many other things these days, they draw you in with a cheap base and then up the price for everything else. Get them second user from ebay and if you are willing to import you can get them for 7.74 quid each plus shipping from the US.

Note that you can use SATA drives without interposers in the MD1000 but are limited to a single 4 lane connection (up to 1.2GB/s). If you use an interposer you can use both 4 lane connections but are limited to 2TB SATA drives. Found this out the hard way :(. The interposers are pricey even second user. Raid 5 or 6 will slow your drives down a fair bit but the 9280 is a pretty decent card.

I would echo what Little_Crow said about having a BBU for any type of parity raid. Not also that BBUs are a disposable item. THey need to be replaced every few years. The other alternative is a card with FBWC (Flash Based Write Cache) which doesn't need replacing. Saying that, the BBUs are not very expensive on ebay but make sure you get the cable and mount too for the first one.

RB
 
Wow, thanks for the info! :)

Noted on the server chassis. I did end up ordering a short depth 4U chassis, but also finally decided on taking the plunge and purchasing a 24U acoustic rack cabinet with 800mm depth rather than fart about with this thing. Will have 3U extra room too, and hoping the 120mm fan in the 4U chassis will be one less noise source to worry about. Tempted to send the 4U case back if I change my mind though... we'll see.

RimBlock said:
Note that you can use SATA drives without interposers in the MD1000 but are limited to a single 4 lane connection (up to 1.2GB/s). If you use an interposer you can use both 4 lane connections but are limited to 2TB SATA drives. Found this out the hard way :(. The interposers are pricey even second user. Raid 5 or 6 will slow your drives down a fair bit but the 9280 is a pretty decent card.
Click to expand...
Bought a bunch of interposer caddies for £27 each, they are on the way as we speak. But I had no idea the interposers weren't needed... All the websites/vendors out there say that they are, which is why I bought them rather than the simple tray ones :confused:

But are you saying that if I intend on having 4TB drives they are useless? :eek: I'm not massively bothered by the speed aspect if I'm honest, so long as I have the storage and data integrity from the RAID 6 and being set up properly... would I be better off taking the interposers off and selling them?

Looked around on the web and can't seem to find anything which makes this completely clear for my particular combination of things, i.e. 4TB SATA drives, daisy chained across 3x MD1000.

So yeah I was wondering, am I being too ambitious to expect 3x MD1000's to be able to run 45x Western Digital 4TB Red drives, daisy chained off a single mini-sas connection from the 9280? (assuming the MD's are running latest A04 firmware), or am I just being paranoid :o

RimBlock said:
I would echo what Little_Crow said about having a BBU for any type of parity raid. Not also that BBUs are a disposable item. THey need to be replaced every few years. The other alternative is a card with FBWC (Flash Based Write Cache) which doesn't need replacing. Saying that, the BBUs are not very expensive on ebay but make sure you get the cable and mount too for the first one.
Click to expand...
Going to buy an LSIiBBU08 which apparently can mount directly on the card.
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom