IO for Home Server, how much?

[mikeh501]

I'm putting together some ideas for a new setup in my house which revolves around the concept of being able to record/play/stream any content (movies,tv,pics,music) to any TV in my house via the network. The crux of it is to bring together all my content, into any room and to rip out SkyHD and go with a no-sub setup. With this, I should be able to record something in one room and playback in another etc.

The one item which isnt obvious off the shelf is what kind of server system I should be speccing for this setup. Heres the requirements...

* Up to 4-5 PC's (acer revos running Win7MC) will be streaming TV (SD/HD) content or SD/HD files (MKVs/AVI's) from the server. TV is streamed to the MC machines via IP on a virtual TV tuner (DVBLink) from the server
* The server will be doing download duties, circa 500-750kbps
* The server will contain at least 4, possibly 8 TV tuners (some cards have 2 tuners, some even 4), all capable of HD either in terrestial or digital flavours.
* The tuners could be simultaneously recording SD/HD content to the disk subsystem.
* The server needs at least 2tb, and for a little future proofing maybe 4tb. I might revise this upwards if I rip all my DVD's to it possibly.
* The server needs to be as small as possible. I dont want a huge tower case.
* The server needs to be low power and quiet.

Now what gives me significant pause for thought it the potential IO requirements i'm generating here and how can I provision it.

Some thoughts...

* Do I get a dedicated NAS box (poss with iSCSI) for the storage and provision a normal desktop type PC for TV tuner duties? Am I asking for trouble with IO doing this?
* Do I just get one machine which does it all, but getting one which has enough PCI slots, enough SATA ports and is still small/quiet is going to be challenging.
* Do I need a decent hardware RAID card? I'm thinking of chucking in as many SATA disks (dont have to be huge) and then configing RAID10 on it for good IO. How much does a card like this cost? What can be done in Win2k8?

Any help gratefully received

Should be an interesting project.

 

[bigredshark]

Well, my first thought was you won't be having huge IO issues with a home server but having read it through...

If you're planning on records up to 8 streams and also playing back 4 streams simultaneously that could be a serious amount of IO depending to the stream bandwidth. I'd say you want either a hardware RAID card or a high performance professional grade NAS device as a minimum and you'll have to work out your performance requirements to guide your disk setup. Even 4TB could be pretty small if you start recording and archiving much HDTV...

IF you do go one device, there's a nice new Fractal design case out which takes 6 disks and a mini-itx motherbaord while looking smart and still being fairly small. Anything with 6 disks in it won't be silent though so a out of the way cupboard isn't a bad bet...

 

[mikeh501]

Thanks for the (only) reply

My suspicion is also that it could be quite a lot of IO, but I guess i'm looking for a steer on just how much, and how it could be provisioned in a small case.

What about RAID cards? i'd like to have some kind of RAID if only to ensure data is spread across many spindles, but i've no idea what is a good one at the home user end of the market. Maybe just go with software raid in win2k8?

The fractal case does look good...

 

[bigredshark]

Well, as a worst case my 1080p content has a bit-rate of around 24,000kb/s (about 12GB an hour, you could go higher if you wanted), that's a worst case maybe but we'll run with it, that means that 8 record streams and 4 playback streams will be a total disk IO of 288Mb/s which is 40MB/s for in the region of 160 IOPS with maximum block size of 256kb (under windows).

That's fairly scary stuff for home storage because of the IOPS requirement, given that we're talking about 12GB per hour here I'd guess that 2TB is the absolute minimum amount of storage you'd want (and more likely 4TB). I'd avoid a NAS as unless it's decent you'll have IO issues and you'll need semi decent network gear into the bargain so built in storage makes more sense.

Depending on your budget either 4x 2TB drives in RAID10 (ideal but expensive for the capacity) or 5 or 6 disks in RAID5 (greater capacity but the IO is right on the edge in throughput and IO). I'd use RAID10 I think.

Either way you need a hardware RAID care, software raid won't cut it at all.

You also face a nasty choice on drives, a server with four+ drives on constantly will consume a fair amount of power, low power drives will help but will be slower...

It's a really interesting project though!

 

[ecksmen]

I think you need to stop thinking about raid and use independant disks for roles, maybe raid 1 them for redundancy.

Something like this -

2 x for temp records
2 x for newly recorded
2 x for old archived.

You can then select high speed disks for temp, newly recorded and then push off to archive.

This doesn't really fit with your must be small, queit and efficent but I think trying to put it all onto 1 array won't get the best out of SATA where random I/O is going to be the main issues with multiple users / recordings.

 

[bigredshark]

I think you need to stop thinking about raid and use independant disks for roles, maybe raid 1 them for redundancy.

Something like this -

2 x for temp records
2 x for newly recorded
2 x for old archived.

You can then select high speed disks for temp, newly recorded and then push off to archive.

This doesn't really fit with your must be small, queit and efficent but I think trying to put it all onto 1 array won't get the best out of SATA where random I/O is going to be the main issues with multiple users / recordings.

I'd say that's a very bad idea, it's artificially limiting in that you have to move content between drives manually or with scripts, a single drive on RAID1 is going to struggle for the IOPS if you're recording/streaming multiple HD files and it's generally inflexible.

A single RAID volume for all the content is more flexible, makes the necessary performance easier to achieve. It's full of upsides and short of downsides, unless you're on a tiny budget and can't afford a decent RAID card, which I'm going to guess isn't the case here.

And buying 'high speed' disks won't help you here, the difference between the best and worst mechanical drives for recording and playing back video will be next to nothing - and contrary to what you've said, pooling the IO in one RAID array is absolutely the best way to improve performance.

 

[AbsenceJam]

I would forget small, low power and quiet because you're going to have a lot of drives and cards (tuners and a hardware RAID). If you're building a network like this then surely you can tuck the server away somewhere.

You can do it with the one machine, just get a decent card from someone like Areca running running RAID 6.

 

[mikeh501]

Thanks for the really valuable feedback guys, especially the fag packet IOPS thinking this is illuminating.

Regarding the case, then yes its a bit of a punt to see if its possible; and i'm fully aware I may have to give up this requirement I do have somewhere to stick it etc.

So where we have got to is that were talking about a single box, not splitting up the tuner duties from the storage duties. This means it cannot be atom based because I need lots of PCI slots and it means a larger chassis given the number of drives. It also means a NAS is out just on the basis of likely IO issues. I think i'd have to spend lots of cash to get something half decent 4 bay ++.

Next question....

I need a hardware RAID card and some SATA drives provisioned into RAID.

* What RAID card? SATA; yes. 4 ports a minimum i'd suspect. Do any do funky drobo raid type stuff? (I have a FW drobo and like it etc)

* What RAID configuration? i've read that some 4 port cards (Adaptec 2410SA) do R10 but surely that isnt enough drives? Working on the basis of min 2tb, mor likely 4tb+. Also read that the Adaptec 2410SA has a 2tb limit! Thats not cool!

 

[AbsenceJam]

You might be able to get away with an mATX board depending on the tuners really.

RAID 10 is pretty wasteful, I would use RAID 6, once you start increasing the number of drives performance is great. Have a look into the Dell PERC cards, you can get them very cheap and they have 8 ports, the 5/i is good (rebranded and flashable LSI) but I don't know about the 6/* ones. A load of the £95~ 2TB F3 EcoGreens on one of these and you'll have a nice array.

Think a bit about reality before you do this. Are you actually going to have all those Revos and are you actually going to be needing so many tuners?

 

[bigredshark]

If you need beyond 4 drives then

http://adaptec.com/en-US/products/Controllers/Hardware/sata/value/SAS-3805/

or this slighty mad option

http://adaptec.com/en-US/products/Controllers/Hardware/sata/value/SAS-31205/

Would suit but they are SATA/SAS so aren't the cheapest on the planet, the convergence of SATA and SAS means there aren't many cheap SATA controllers with more than 4 ports, your best bet might be the on board on a decent motherboard...

As to number of drives, well, 4 2TB drives in RAID10 would do the job in terms of performance and would give 4TB of usable space.

I don't know of any hardware card which does drobo like functionality - if anybody else does I'd be interested to hear too...

 

[bigredshark]

RAID 10 is pretty wasteful, I would use RAID 6, once you start increasing the number of drives performance is great.

RAID6 is disk for disk the worst performing of any common RAID type, it's great for some applications because it's buying you good resiliency and decent capacity but fast it most definitely isn't. The only decently fast parity based RAID system is Netapp's proprietary RAID-DP and they cheat...

To get up to my guesstimate (and it's a wild guesstimate I freely admit) you'd need at least 6 disks in RAID6 and that's still less IOPS than 4 disks in RAID10. That figure will drop again if you go for low power drives like Barracuda LP or WD Green editions...

 

[ecksmen]

I'd say that's a very bad idea, it's artificially limiting in that you have to move content between drives manually or with scripts, a single drive on RAID1 is going to struggle for the IOPS if you're recording/streaming multiple HD files and it's generally inflexible.

A single RAID volume for all the content is more flexible, makes the necessary performance easier to achieve. It's full of upsides and short of downsides, unless you're on a tiny budget and can't afford a decent RAID card, which I'm going to guess isn't the case here.

And buying 'high speed' disks won't help you here, the difference between the best and worst mechanical drives for recording and playing back video will be next to nothing - and contrary to what you've said, pooling the IO in one RAID array is absolutely the best way to improve performance.

The whole point of my post is to consider multiple volumes -however decided to be split up there's little point in having high speed raid 10 with lots of waste for content that never gets watched. One RAID array may be flexible, but it's a waste.

 

[SimonR]

Firstly what OS are you thinking of running for your server. If it is WHS then forget any talk of RAID, it doesn't sit well with WHS.

Secondly with the demands of that system, I seriously doubt you are going to get anything particularly low powered.

Thirdly I would have a trawl of both The Green Button (Media Center Edition) and We Got Served (Windows Home Server) websites. There are plenty of people on those forums that have similar setups, that you describe running day in day out and have gone through all of the pain making them work reliably. Most of them are US based and running ATSC tuners, but the concepts and most of the hardware are identical.

One of the problems that you will face, particularly if you want a low power set-up is finding a suitable mobo with sufficient PCI-e slots to acomodate all of your tuners.

Fortunately something like http://shop.blackgold.tv/epages/BT3159.sf/en_GB/?ObjectPath=/Shops/BT3159/Products/BGT3595 when they become available to retail should be ideal for your requirements.

I think W7 MCE supports upto 4 tuners of each kind, so using those black gold cards, you could actually have say 4 dvb-t and 4dvb-s tuners, running 4 cards. A word of warning though, although they do not draw a huge amount of power, they do get hot so you better have plenty of cooling in whatever case you choose.

So there goes your idea of not using a large case. Also from the experiences of other forum members, your 2 TB of storage will not last you very long, especially if you are recording a lot of HD streams.

One solution could be to build a relatively low power MCE 7 machine using an atom motherboard and one of those black gold tuners for satelite, use something like a pair of of DVB Link (Homerun) network tuners for dvb-t and use a QNAP box as a HD subsystem running on a gigabit network.

That way the tuners and the NAS could be hidden away and all you would have is a small MCE box on display, with 2 satellite cables going into it.

Mushii

 

[mikeh501]

Firstly what OS are you thinking of running for your server. If it is WHS then forget any talk of RAID, it doesn't sit well with WHS.

I'm thinking Windows 2008 Server Standard R2 and set it up with AD etc. One of the questions i've got is driver compatability with the tuner cards but thats just a google for another day.

Secondly with the demands of that system, I seriously doubt you are going to get anything particularly low powered.

yeah, i think i've come to that realization.

Thirdly I would have a trawl of both The Green Button (Media Center Edition) and We Got Served (Windows Home Server) websites. There are plenty of people on those forums that have similar setups, that you describe running day in day out and have gone through all of the pain making them work reliably. Most of them are US based and running ATSC tuners, but the concepts and most of the hardware are identical.

Good idea

Fortunately something like http://shop.blackgold.tv/epages/BT3159.sf/en_GB/?ObjectPath=/Shops/BT3159/Products/BGT3595 when they become available to retail should be ideal for your requirements.

The BG3595 is definately top of my list, but I still need to do more research. Wonder if the drivers work in WinSvr08....

I think W7 MCE supports upto 4 tuners of each kind, so using those black gold cards, you could actually have say 4 dvb-t and 4dvb-s tuners, running 4 cards. A word of warning though, although they do not draw a huge amount of power, they do get hot so you better have plenty of cooling in whatever case you choose.

I want to run all the tuners in a single box and then have them shared over IP with the DVBLink software. This apparently allows "virtual" TV tuners to appear in the Atom based Acer Revo running win7 MC without them actually having to be present in hardware and can be run remotely to the server. Therefore the server does all the hard work, and importantly I can record in one room and playback in another which is one of my requirements.

Also from the experiences of other forum members, your 2 TB of storage will not last you very long, especially if you are recording a lot of HD streams.

Indeed.... hence why I want a decent IO subsystem with plenty of ports; just finding one is expensive!

One solution could be to build a relatively low power MCE 7 machine using an atom motherboard and one of those black gold tuners for satelite, use something like a pair of of DVB Link (Homerun) network tuners for dvb-t and use a QNAP box as a HD subsystem running on a gigabit network.

That way the tuners and the NAS could be hidden away and all you would have is a small MCE box on display, with 2 satellite cables going into it.

If your going to the trouble of implementing the server and DVBlink then why not put the satellite feed into the server too?

 

[AbsenceJam]

RAID6 is disk for disk the worst performing of any common RAID type, it's great for some applications because it's buying you good resiliency and decent capacity but fast it most definitely isn't. The only decently fast parity based RAID system is Netapp's proprietary RAID-DP and they cheat...

To get up to my guesstimate (and it's a wild guesstimate I freely admit) you'd need at least 6 disks in RAID6 and that's still less IOPS than 4 disks in RAID10. That figure will drop again if you go for low power drives like Barracuda LP or WD Green editions...

I meant fast as in fast enough and in itself.

It doesn't really matter for this sort of setup, the realistic amount of disk activity won't be an issue.

 

[bigredshark]

I meant fast as in fast enough and in itself.

It doesn't really matter for this sort of setup, the realistic amount of disk activity won't be an issue.

Would you like to scroll back up and look at my rough IOPS calculations, they're rough but they're not out of the ballpark, somewhere between 125-175 IOPS isn't an unrealistic requirement and using RAID6 with SATA drives means a minimum of 6 drives to get there (and I don't think that's quite enough).

 

[AbsenceJam]

Would you like to scroll back up and look at my rough IOPS calculations, they're rough but they're not out of the ballpark, somewhere between 125-175 IOPS isn't an unrealistic requirement and using RAID6 with SATA drives means a minimum of 6 drives to get there (and I don't think that's quite enough).

I've built and used lots of home media servers like this, it's not a problem, even in software for the most part, your figures are off.

Freeview/Freesat SD is MAX 8Mb, the 3 HD channels MAX 20Mb (can't cap Sky or Virgin HD, also in reality bitrates are a lot lower than this, half or less). And the highest bitrate media you can play back would be Blu-ray at 40Mb.

In reality this potential setup might maybe be recoding 1-2 channels, watching another, maybe playing a BD or DVD and listening to music.

 

[bigredshark]

I've built and used lots of home media servers like this, it's not a problem, even in software for the most part, your figures are off.

In reality this potential setup might maybe be recoding 1-2 channels, watching another, maybe playing a BD or DVD and listening to music.

He said up to 8 tuners, simultaneously recording HD or SD content (and with dvb-t2 online and tuners bound to turn up soon that's not impossible). The bitrate for HD content is a reasonable top end figure. My figures are spot on for the information provided.

Maybe it'll be used less, but given the requirements posted you don't have enough disk IO available with SATA and RAID6 short of a stupid number of disks

 

[Yamahahahahaha]

You definitely need a RAID card for this. Along with the Areca cards, you should be able to pick up cheap Dell PERC 5/i or PERC 6/i cards off Ebay (based off an LSI design, LSI MegaRAID SAS 8408E for the Perc5 and MegaRAID SAS 888 for Perc6 iirc). These take up to 8 drives.

You need RAID10 with at least 4 drives.

A fully uncompressed 1080p video can hit about 135mbit/s, so if you want to leave yourself lots of leeway, work with that figure.

 

[n3vrmind]

Surprised no one has suggested what seems to be obvious to me

Tier1 Storage: SSD disks, used for the recording
Tier2 Storage: Bulk Sata

If your writing 8 streams at once, you will have 8 very fragmented files. However if each time a recording is completed it is then moved over to the Tier2 storage it will be rewritten as a cotinguous file and reduce the chances of stuttering during playback

You can use something as simple as a robocopy job to shift files, robocopy can automatically skip in use (i.e. files still being recorded) so you could have it run every 15 minutes as a sheduled job to catch and move any completed files

I already take this approach with torrents on my home server, they are written to one drive while in progress and then utorrent is configured to move them to another drive when completed. Its better this way because if you have a slow torrent that will take a week or more downloading to a drive with a spindle that drive is never able to spin down and consumer drives are just not designed for 24x7 use.