Network design for a home renovation.

[Zarf]

Needlessly complicated and expensive, The weak point you are trying to overcome is poor possible poor performance from the NAS when dealing with multiple streams. If you run a decent NAS system like Openfiler using RAIDZ It'll cope fine with 3x sequential 1080p streams.

A single 1Gbit link is plenty, 3 sequential 1080p streams won't even use 10MB/s, and I'll be surprised if your backup NAS box writes at more than 30MB/s.
Don't bother with two subnets either, it's just added complication if you want to get on the net from your second subnet. Just get yourself two unmanaged 8ports and run a single subnet.

I'm running a similar setup to that a home (Using Windows Server 2003 on file server though), just run a quick test and I'm not having any problems playing four different 1080p 8GB MKVs on my desktop,another two on my laptop and two more on my HTPC simultaneously. I'd probably be able to do more but I'm out of CPU cores

 

[RimBlock]

Wow, quite a lot to discuss.

You would be using port-based VLANs, whereby each port can be a member of more than one VLAN at a time. You would assign the NAS ports 23/24 to be members of both VLAN A and VLAN B.

Ahh, so ports can be assigned to multiple VLans. Makes perfect sense.

Having said the above, I don't see the point of different subnets anyway. Traffic will already be getting separated in the NAS by addressing the different shares i.e. the 1.x subnet devices will only be accessing Share 2 (and NIC 2 if you abandon bonding). As others have said, you could simplify a lot.

Yep, am coming around to agreeing with that .

In terms of disk performance the streaming will not tax a single HDD. I've seen benches showing approx 40MBps random access read speed for the older version of that drive (4x 500GB platters, now shipping as 3x 667GB which should be faster). I can't state for exactly your scenario, but suggest you look at PVRs. The relatively old 160Gb IDE HDD in a Virgin V+ box can manage to record 3 streams while playing 2. Granted this is heavily compressed broadcast HD (approx 10Mbps) and your MKVs will be around double that, but WD do quote the AV version of your drive as capable of playing upto 12 HD streams. I think the only thing that will stress a single HDD is computer file use e.g. copying large files or making backups.

Sure but if I am going to have a backup drive, why not get it working too . Interesting point about having a separate upload drive.

My understanding is that some RAID1 implementations can read from both disks when there are multiple read requests. Alternatively R5 could work for you, but take care over write performance. Soft RAID (i.e. OS or desktop motherboard) implementations generally have a heavy penalty. Ideally go for a HW controller with dedicated XOR chip e.g. Dell PERC, HP SmartArray. Although, for the same cost you could invest in a 4th drive and just use RAID 1+0 with soft RAID. Whatever, make sure the controller and OS driver make good use of NCQ, as multi-user scenarios is where this shines.

Adding a dedicated RAID card would be great but the cost is just going up and up at the moment. It can be an upgrade at a later time though.

The NAS RAM is overkill at 6GB if all it's doing is file sharing. You could probably drop to 2GB with no visible penalty.

I already have that machine and it already has that RAM in it. I was using the machine for virtulization hence the high ram. My own desktop already has 8GB which is its max so no point swapping it.

My suggestion:
Use a single subnet, no VLANs. Keep the 24port smart switch. Keep NIC bonding. Use 2 HDDs in NAS, but not R1. Have all streaming devices use HDD1. All computer devices and video uploading goes to HDD2, which syncs automatically to HDD1 during off-peak hours.

This looking any better. Most of what you suggested with a bit more of a twist on the NAS.

[posted in previous reply]

Thanks for taking the time. Very helpful as are all the responses here.

RB

 

[RimBlock]

Hi,

Just for a little more backup info, here is what I currently have in the place we have just sold.

Note: I have expanded the IPTV side of things and I got the name of the D-Link switch completely wrong (down to even the manufacturer ). I have updated it in this diagram.

Much better, but I'd still be looking to get rid of that d-link switch and ISP provided router for something better and perhaps an all in one. I'd also move the backup NAS onto the procurve. ]

Yes, agreed - much better.
I think I'd run DHCP & DNS on the Linux server. Of course, it does depend how much access RB has to the ISP-supplied router. Being Singapore, I wouldn't be surprised if it was compulsory and locked down.

The ISP supplied router is not locked down at all. I can access the web interface, play with the firewall, turn off the wireless etc without any issues. The problem is that the ADSL router they supply deals with the VIOP, IPTV (21Mbps min req for HD) and the internet feed (only 8Mbps at the moment). This is also the only device that is supported by the ISP so if something goes wrong with any of those services and I have changed the ADSL router, they will not support it. THe IPTV has had problems quite a few times due to low available bandwidth on the copper line in to my apartment so I am cautious at replacing it.

The D-Link switch is really a Linksys router with dual band wireless and is the Linksys top of line (my fault for getting the model wrong ).

Running the DHCP on the Linux box (NAS or backup NAS ?) is certainly an option. I use openDNS for my DNS requirements as Singapore does seem to block a few sites via their ISPs removing the DNS entries in local DNS servers.

Does seem a bit of a shame to spend all that money on a managed switch but if you're set on bonding then fair enough.

I'd also look to have a decent DHCP / DNS server internally for all that, but thats just me.

I am not so set on bonding but just want to make sure both network cards are utilized. If they are not set with each one routing to a separate subnet and they are not bonded then how to ensure both are used ?

Needlessly complicated and expensive, The weak point you are trying to overcome is poor possible poor performance from the NAS when dealing with multiple streams. If you run a decent NAS system like Openfiler using RAIDZ It'll cope fine with 3x sequential 1080p streams.

I have just installed OpenFiler in a virtualbox VM. I have just done a basic setup but am not able to access any of the shares. Have set shares, share folders, groups, users, share access to groups, added my workstation in on networks but cannot get to it. As it was midnight by this that I stopped. The VM network is bridged and the DHCP ip for the VM is on the same subnet as my workstation. I have a reasonable knowledge of Linux but getting it setup with very little documentation means I will need to take a bit more time on this, especially if Virtual Box is throwing in some extra issues.

I am using virtual box rather than the machine I will be using for the NAS server as that machine is boxed up for the house move and I am coming to the UK tomorrow for 3 weeks hence I will not unpack it until I return to Singapore in 3 weeks even though I would love to do so now .

How do you have RADZ setup, what config ?

A single 1Gbit link is plenty, 3 sequential 1080p streams won't even use 10MB/s, and I'll be surprised if your backup NAS box writes at more than 30MB/s.

What about copying to the NAS at the same time and internet connectivity ?

Don't bother with two subnets either, it's just added complication if you want to get on the net from your second subnet. Just get yourself two unmanaged 8ports and run a single subnet.

The two subnets were to allow allocation of one subnet to each of the NAS nics. Using bonding will mean I do not need to do that so no subnets needed.

Two 8 ports will at best (only one server NIC) will not give any expandability and if I keep two nics on the server then I will be 1 port short.

I'm running a similar setup to that a home (Using Windows Server 2003 on file server though), just run a quick test and I'm not having any problems playing four different 1080p 8GB MKVs on my desktop,another two on my laptop and two more on my HTPC simultaneously. I'd probably be able to do more but I'm out of CPU cores

So streaming should be fine with one NIC. What about when you are also copying an 8GB file to the NAS at the same time ?

If I can utilize both server nics without the need for subnets and bonding then can anyone suggest how to do it and a cheaper replacement for the HP Procurve that would fit the bill. I can get the Procurve for around 200 Quid (GBP)

One problem I have at the moment with my current setup is that if the nas is running a torrent client then the media player which connects wirelessly usually cannot connect to the network. The XBox also has the same issue. If I stop the torrent client (transmission) then I have no issues. The client is very rarely maxing the download speed of the internet connection but may be maxing the upload. This is why I am keen to put the backup NAS on the ADSL router and away from the main switch.

Thanks all
RB

 

[Zarf]

How do you have RADZ setup, what config ?

Not actually running it myself, as I have a hardware RAID card (Dell Perc 5/i) and a Windows Server 2003 setup. I have heard good things about it though, but Openfiler might be a bit complicated to set up in a home environment (iirc it wants a proper domain setup). FreeNAS also supports ZFS and is easier to set up.

What about copying to the NAS at the same time and internet connectivity ?

http://img526.imageshack.us/img526/4071/netstress.jpg
I get around 16MB/s writing to the NAS at the same time as it dishes out 4 * 1080p streams, which isn't too bad. Really though you should schedule your backups for the early hours of the morning when there is almost no other activity.

The two subnets were to allow allocation of one subnet to each of the NAS nics. Using bonding will mean I do not need to do that so no subnets needed.

Even without bonding you can have two nics in a single PC on a subnet, just assign them different IP addresses.

Two 8 ports will at best (only one server NIC) will not give any expandability and if I keep two nics on the server then I will be 1 port short.

If you need expandability in the future, you just grab another 8 port switch and daisy chain it in. They are only around £20 a pop, much cheaper solution per port than getting a 24 port.

One problem I have at the moment with my current setup is that if the nas is running a torrent client then the media player which connects wirelessly usually cannot connect to the network. The XBox also has the same issue. If I stop the torrent client (transmission) then I have no issues. The client is very rarely maxing the download speed of the internet connection but may be maxing the upload. This is why I am keen to put the backup NAS on the ADSL router and away from the main switch.

Thanks all
RB

Only thing I can think of is that your router is terrible and is maxing it's CPU/RAM out managing torrent connections, and thus isn't serving DHCP addresses properly. What happens if you reduce the number of torrent connections (which would probably boost your speeds too if your router is limiting you) and/or use static IP's?

 

[vsmora]

Liking the latest revision much more. For the NICs, as stated you don't need to have bonding to utilise both. The left-hand devices can simply get to the NAS via 192.168.1.1 and the right-hand via xxx.2 and despite sharing the same subnet, the traffic will travel via the correct NIC. Also note that you don't necessarily need a smart switch to utilise bonding. Haven't setup a Linux NAS yet but on the Win servers I have done, which had 2-4matching NICs, the manufacturer (e.g. HP/Dell/Intel) supplied a suitable driver and utility to enable them to emulate a single virtual NIC in the OS. The NICs can then be plugged into any standard unmanaged switch, so not reliant on LACP. It's possible your OS might offer NIC teaming natively. The reason I suggested keeping the smart switch is because they have better features, lifetime warranty and crucially build quality that you won't get from a cheapo £20 8-port. The Netgear GS724T is on promo and can be had for ~£175 (with a free fibre module that you could maybe sell to recoup some value).

The HDD/NIC layout in the NAS is the only thing slightly concerning me now. Not because of performance, more to do with setting up and managing everything e.g. setting up the right bindings and handling syncing with a drive that's significantly smaller than the main drives. If you're confident that you can handle it and the benefits are worth it then I guess you're good to go.

 

[tbz_ck]

One problem I have at the moment with my current setup is that if the nas is running a torrent client then the media player which connects wirelessly usually cannot connect to the network. The XBox also has the same issue. If I stop the torrent client (transmission) then I have no issues. The client is very rarely maxing the download speed of the internet connection but may be maxing the upload. This is why I am keen to put the backup NAS on the ADSL router and away from the main switch.

Thanks all
RB

I've seen this behaviour on some routers where an internal IP table (storing info on current connections) maxes out the available RAM forbidding more connections until others time out. I had this on a Zyxel HW660 in particular.

I'm liking the simplified layouts of the network design. It looks like you don't need the initial complexity you thought you would seeing as those with experience of multiple HD streams still can't max out disk or network connections. In which case the simpler the better, and cheaper!

 

[RimBlock]

Not actually running it myself, as I have a hardware RAID card (Dell Perc 5/i) and a Windows Server 2003 setup. I have heard good things about it though, but Openfiler might be a bit complicated to set up in a home environment (iirc it wants a proper domain setup). FreeNAS also supports ZFS and is easier to set up.

From a bit of reading it seems that ZFS is kneecapped in Linux but has a full implementation in BSD and Open Solaris. Whilst I work on Solaris 8&10 daily at work, I would rather not have it at home as I am much more used to Linux (RHE/CentOS) than Solaris from an SA point of view. Does look good though with RAIDZ2.

Really though you should schedule your backups for the early hours of the morning when there is almost no other activity.

Ideally yes but the uploads will be manually instigated. I guess I could 'tag' items to be backed up and have a nightly job copy the tagged files over only.

Even without bonding you can have two nics in a single PC on a subnet, just assign them different IP addresses.

Doh.. Yes of course I could. I blame old age for not realising that. All the traffic from the NAS will be in response and if it comes in on NIC-1 then all the responses will go back on the same NIC.

If you need expandability in the future, you just grab another 8 port switch and daisy chain it in. They are only around £20 a pop, much cheaper solution per port than getting a 24 port.

Yeah but still prefering the more robust 24port switch. Have had a procurve (8 port) in the past and had 0 problems with it.

Only thing I can think of is that your router is terrible and is maxing it's CPU/RAM out managing torrent connections, and thus isn't serving DHCP addresses properly. What happens if you reduce the number of torrent connections (which would probably boost your speeds too if your router is limiting you) and/or use static IP's?

Hmm, top of the range for Linksys home routers. Would be very sad if it really was that bad .

Liking the latest revision much more. For the NICs, as stated you don't need to have bonding to utilise both. The left-hand devices can simply get to the NAS via 192.168.1.1 and the right-hand via xxx.2 and despite sharing the same subnet, the traffic will travel via the correct NIC. Also note that you don't necessarily need a smart switch to utilise bonding. Haven't setup a Linux NAS yet but on the Win servers I have done, which had 2-4matching NICs, the manufacturer (e.g. HP/Dell/Intel) supplied a suitable driver and utility to enable them to emulate a single virtual NIC in the OS. The NICs can then be plugged into any standard unmanaged switch, so not reliant on LACP. It's possible your OS might offer NIC teaming natively. The reason I suggested keeping the smart switch is because they have better features, lifetime warranty and crucially build quality that you won't get from a cheapo £20 8-port.

Yes, I would agree and this is why I like the idea of geting the procurve.

The Netgear GS724T is on promo and can be had for ~£175 (with a free fibre module that you could maybe sell to recoup some value).

Interesting. Thanks, I will take a look.

The HDD/NIC layout in the NAS is the only thing slightly concerning me now. Not because of performance, more to do with setting up and managing everything e.g. setting up the right bindings and handling syncing with a drive that's significantly smaller than the main drives. If you're confident that you can handle it and the benefits are worth it then I guess you're good to go.

Uploads will go to the 500GB drive and then will be automatically distributed to the two 2GB disks on a nightly basis. The 500GB will just be for uploads, the OS and maybe DHCP.

I've seen this behaviour on some routers where an internal IP table (storing info on current connections) maxes out the available RAM forbidding more connections until others time out. I had this on a Zyxel HW660 in particular.

Again, that would be very sad if that was the case considering the price of the router. I have another 1Gbit switch I could use to move the NAS off the Linksys and I will see if that stops the issues. Thanks to you and Zarf for the info on that.

I'm liking the simplified layouts of the network design. It looks like you don't need the initial complexity you thought you would seeing as those with experience of multiple HD streams still can't max out disk or network connections. In which case the simpler the better, and cheaper!

Yep seems to be the case and what is better is with the 24port business grade switch I can modify the infrastructure quite easily as more requirements come up.


Thanks again all.

RB

 

[encephalopathy]

If you need expandability in the future, you just grab another 8 port switch and daisy chain it in. They are only around £20 a pop, much cheaper solution per port than getting a 24 port.

it's a pain though as each switch needs a powerbrick and socket so you end up with more wires/sockets used that with a single bigger switch at the start

 

[RimBlock]

Ok, moving on slightly.

I have bought a couple of Intel 1000/pro network cards, an Icydock 5 drive cage and an adaptec 1405 RAID card for a Raid 5 implementation (3 data & 1 parity). The last bay for the Icydock will be for the boot drive at this point.

I will populate the array with 1.5TB WD Caviar green drives (I have one already) and the boot drive will probably be my Hitachi XXXGB drive in my desktop machine as I have just got a SSD for it.

I also got another WD HD Live TV. I have the AC Ryan device but I find it is not so good with playback high bandwidth HD content. I seem to get lines like interlacing artefacts we used to see on badly encoded VHS copies. I read that the problem could be due to the Playon HD using the same chip for video as it uses for networking. The WD does have problems with high bandwidth HD as well but it will skip frames and the problems seem to not be so common. Shame as the Playon has a much better interface.

The HP switch prices are up to around 300quid now so I need to shop around a bit more to see if I can find it cheaper.

Cheers
RB

 

[RimBlock]

And it changes again... for the NAS.

The Adaptec 1405 is a HBA even though it was advertised on a large on-line store as being a RAID card . Will teach me for believing what the seller is saying.

I now have;
5x WD-CG 1.5TB drives in software Raid5.
350GB boot drive
250GB upload drive (possible to be sold)
Adaptec 2410SA PCI-X raid card (4 ports) - does not see drives over 1GB .

I am really not having any sort of luck with Raid controllers so I will stick with software Raid for now and use the 2410 with my Windows desktop and my Vertex II drives.

I have ordered a cable fro the Adaptec 1405 HBA as I have run out of ports (6) on the motherboard and the cable can be used for a better adapter if I choose to upgrade later. The HBA will be used for the boot / upload / non raid other share drive. The motherboard will have the 5 raid drives and the dvd rom drive.

Only 2-3 weeks to go before seriously looking at buying the networking stuff and starting an install.

RB