Network design for a home renovation.

[RimBlock]

Hi,

After a bit of discussion here, I have now decided to install cat6 as I renovate the new two storey apartment I am in the process of buying.

The network(s) I plan to put in will be used for media (HD, SD and music) and for the internet.

I am trying to network most rooms with two points (one for iptv they have here and the other for media players streaming from a NAS box).

As you can see from the diagram below I am trying to maximise the throughput from the NAS by having a separate network card for each subnets dedicated to AV and by having a share based around a individual HDD dedicated to each NIC. The theory is that if two people are streaming HD content on subnet A then they will not affect anyone on subnet B from doing the same.

Internet access is limited to my backup NAS box which will also perform download duties if required, our two PC's and my home and work laptops.

I have not connected the other devices to the internet feed as I do not want the switch supplying the feed to bridge between subnet A and B thus negating the point of the NAS having separate NICS for each subnet.

I would love to hear any ideas for improving on the design if anyone has any.

Thanks
RB

 

[RimBlock]

Ok, first revision. No reason why subnet A cannot have internet access as long as it does not bridge to subnet B

RB

 

[RimBlock]

Thanks to both of you.

I initially thought of a 24 port switch but without the separate subnets the traffic from the server would most likely only use the single NIC. Single point of failure is not really an issue as it is just for movies etc at home.

Setting up two vlans is something I will take a look in to. Good idea.

I also looked at link aggregation and would love to do it but the 24port switches tend to be quite pricey. I can get a couple of TP-Link Gbit 8 port switches for around S$200. Can I get a Gbit 24 port HP Procurve for under S$400 ? I see there is some recon stuff on t'bay but am not sure which have link aggregating.

The switches will all be fed via a patch panel (well two, one for each subnet).

The other idea about the having the two independent disks is to not let multiple requests all trying to access different parts of the disks to slow the disk down two much as it jumps back and forth. The two disks would also be a backup of each other should one fail.

RB

 

[RimBlock]

Ok, seems like the HP Procurve 1810G-24 has 22 Gbit ports, 2 Gbit/GBIC uplink ports, supports VLAN and is web managed and can be had for 240quid or so.

Looks like it may fit the bill. Any other models worth looking at ?

Thanks
RB

 

[RimBlock]

New version (with LACP) as it was done before I saw your reply.

A bit more detail on the components as well.

The NAS shares will be linked to the VLans via a login / password setup.

Still trying to get my head fully around the VLan.

If the NAS NICs are aggregated, how would the data get to whichever VLan is not on the servers subnet/same VLan as the server ?. Would the switch act as a router between VLans and any connections not on a VLan ?.

New diagram

Many thanks
RB

 

[RimBlock]

Thanks for the advice.

I am working towards allowing the following to run;
3*8GB MKV HD streams from say, living room, bedroom and study (NAS shared not transcoded and streamed).
200MBit Internet connection
Copying a large 8GB MKV Bluray rip from PC1 -> NAS

The problem with Raid 1 is that all requests for data will be hitting one disk. If the disk can provide 744Mbps (average) for a single stream then what will this drop to trying to manage 4 streams (3 HD reads and 1 HD write). Will it drop to a level where it cannot share the HD streams quickly enough for playback by the media players. Is there any room for larger bandwidth ? Two drives acting independently should half the load (all things being equal), allow a faster copy to the NAS.

I really have no idea of the answer to the above questions and would be very interested to hear from others more knowledgeable.

I am not bothered about internet connectivity to the second VLan as all it will be used for is updating firmware on the media players at this point.

Another option would be looking at going for Raid 5 so the media disks would be striped allowing faster throughput to better try to match the capacity of two NICS. The third drive would be used for parity.

For an understanding of volume, I have around 2TB of data at the moment.

Drive failure will be a pain but protecting against it is not a major concern as I can always re-rip my DVD's and Blurays if needed.

Thanks
RB

 

[RimBlock]

A couple of points:

I may have missed this but is there a DHCP server (the Procurve?) or are you using static IP addresses?

I wouldn't advise RAID5 using only three HDDs - the performance degradation when one fails is enormous. I'd go for four or, preferably, five drives.

Yep, you are quite right. I thought about the DHCP server and then promptly forgot about them .

At this time the ADSL box provides DHCP functions. If I kept the VLans then I would probably have another unit on the second lan act as a second DHCP server.

3 drives will max the 2Gbit aggregated NIC throughput based on specs with a bit of extra speed to spare. As it is not a business critical set-up, I think I can live with a bit a slowdown while I source a replacement drive or even cope with being without the NAS for a week or so. How easy is it to expand from 3 to 4 drives in raid5 ? Is it simply a case of adding another drive as a data drive and then rebuilding the raid set ?.

I am of course trying to keep this project to as low a budget as possible. It has gone from S$1K to almost S$2k although the switch accounts for around S$700 of that .

Thanks for the feedback it is very much appreciated.

RB

 

[RimBlock]

Based on what you've said above I'd maybe look at trunking a nice with LCAP and keeping the video on a separate set of drives to be sure of no problems but "8Gb MKV" is not really that high bitrate (native BluRay is what, 55Gb?).

Nope but multiple streams along with internet and moving files around etc. All adds up and would rather over engineer a bit than to finish the set-up and find out the media players are stuttering and sound is cutting out. I can imagine my wifes face if that was happening and then I showed her the S$2k bill .

If it were me I'd personally say leave the media players with internet access, upgrading firmware aside it may be handy for features such as internet radio, or youtube streaming if that sort of thing takes your fancy?

Yep, understood but they are not really of interest to me at the moment. That could have course change in the future.

Cheers
RB

 

[RimBlock]

Ah nuts. I wrote a long reply to Vsmora at work and forgot to post it .

Ok, updated diagram now and I will post the reply from work tomorrow morning.

RB

 

[RimBlock]

Wow, quite a lot to discuss.

You would be using port-based VLANs, whereby each port can be a member of more than one VLAN at a time. You would assign the NAS ports 23/24 to be members of both VLAN A and VLAN B.

Ahh, so ports can be assigned to multiple VLans. Makes perfect sense.

Having said the above, I don't see the point of different subnets anyway. Traffic will already be getting separated in the NAS by addressing the different shares i.e. the 1.x subnet devices will only be accessing Share 2 (and NIC 2 if you abandon bonding). As others have said, you could simplify a lot.

Yep, am coming around to agreeing with that .

In terms of disk performance the streaming will not tax a single HDD. I've seen benches showing approx 40MBps random access read speed for the older version of that drive (4x 500GB platters, now shipping as 3x 667GB which should be faster). I can't state for exactly your scenario, but suggest you look at PVRs. The relatively old 160Gb IDE HDD in a Virgin V+ box can manage to record 3 streams while playing 2. Granted this is heavily compressed broadcast HD (approx 10Mbps) and your MKVs will be around double that, but WD do quote the AV version of your drive as capable of playing upto 12 HD streams. I think the only thing that will stress a single HDD is computer file use e.g. copying large files or making backups.

Sure but if I am going to have a backup drive, why not get it working too . Interesting point about having a separate upload drive.

My understanding is that some RAID1 implementations can read from both disks when there are multiple read requests. Alternatively R5 could work for you, but take care over write performance. Soft RAID (i.e. OS or desktop motherboard) implementations generally have a heavy penalty. Ideally go for a HW controller with dedicated XOR chip e.g. Dell PERC, HP SmartArray. Although, for the same cost you could invest in a 4th drive and just use RAID 1+0 with soft RAID. Whatever, make sure the controller and OS driver make good use of NCQ, as multi-user scenarios is where this shines.

Adding a dedicated RAID card would be great but the cost is just going up and up at the moment. It can be an upgrade at a later time though.

The NAS RAM is overkill at 6GB if all it's doing is file sharing. You could probably drop to 2GB with no visible penalty.

I already have that machine and it already has that RAM in it. I was using the machine for virtulization hence the high ram. My own desktop already has 8GB which is its max so no point swapping it.

My suggestion:
Use a single subnet, no VLANs. Keep the 24port smart switch. Keep NIC bonding. Use 2 HDDs in NAS, but not R1. Have all streaming devices use HDD1. All computer devices and video uploading goes to HDD2, which syncs automatically to HDD1 during off-peak hours.

This looking any better. Most of what you suggested with a bit more of a twist on the NAS.

[posted in previous reply]

Thanks for taking the time. Very helpful as are all the responses here.

RB

 

[RimBlock]

Hi,

Just for a little more backup info, here is what I currently have in the place we have just sold.

Note: I have expanded the IPTV side of things and I got the name of the D-Link switch completely wrong (down to even the manufacturer ). I have updated it in this diagram.

Much better, but I'd still be looking to get rid of that d-link switch and ISP provided router for something better and perhaps an all in one. I'd also move the backup NAS onto the procurve. ]

Yes, agreed - much better.
I think I'd run DHCP & DNS on the Linux server. Of course, it does depend how much access RB has to the ISP-supplied router. Being Singapore, I wouldn't be surprised if it was compulsory and locked down.

The ISP supplied router is not locked down at all. I can access the web interface, play with the firewall, turn off the wireless etc without any issues. The problem is that the ADSL router they supply deals with the VIOP, IPTV (21Mbps min req for HD) and the internet feed (only 8Mbps at the moment). This is also the only device that is supported by the ISP so if something goes wrong with any of those services and I have changed the ADSL router, they will not support it. THe IPTV has had problems quite a few times due to low available bandwidth on the copper line in to my apartment so I am cautious at replacing it.

The D-Link switch is really a Linksys router with dual band wireless and is the Linksys top of line (my fault for getting the model wrong ).

Running the DHCP on the Linux box (NAS or backup NAS ?) is certainly an option. I use openDNS for my DNS requirements as Singapore does seem to block a few sites via their ISPs removing the DNS entries in local DNS servers.

Does seem a bit of a shame to spend all that money on a managed switch but if you're set on bonding then fair enough.

I'd also look to have a decent DHCP / DNS server internally for all that, but thats just me.

I am not so set on bonding but just want to make sure both network cards are utilized. If they are not set with each one routing to a separate subnet and they are not bonded then how to ensure both are used ?

Needlessly complicated and expensive, The weak point you are trying to overcome is poor possible poor performance from the NAS when dealing with multiple streams. If you run a decent NAS system like Openfiler using RAIDZ It'll cope fine with 3x sequential 1080p streams.

I have just installed OpenFiler in a virtualbox VM. I have just done a basic setup but am not able to access any of the shares. Have set shares, share folders, groups, users, share access to groups, added my workstation in on networks but cannot get to it. As it was midnight by this that I stopped. The VM network is bridged and the DHCP ip for the VM is on the same subnet as my workstation. I have a reasonable knowledge of Linux but getting it setup with very little documentation means I will need to take a bit more time on this, especially if Virtual Box is throwing in some extra issues.

I am using virtual box rather than the machine I will be using for the NAS server as that machine is boxed up for the house move and I am coming to the UK tomorrow for 3 weeks hence I will not unpack it until I return to Singapore in 3 weeks even though I would love to do so now .

How do you have RADZ setup, what config ?

A single 1Gbit link is plenty, 3 sequential 1080p streams won't even use 10MB/s, and I'll be surprised if your backup NAS box writes at more than 30MB/s.

What about copying to the NAS at the same time and internet connectivity ?

Don't bother with two subnets either, it's just added complication if you want to get on the net from your second subnet. Just get yourself two unmanaged 8ports and run a single subnet.

The two subnets were to allow allocation of one subnet to each of the NAS nics. Using bonding will mean I do not need to do that so no subnets needed.

Two 8 ports will at best (only one server NIC) will not give any expandability and if I keep two nics on the server then I will be 1 port short.

I'm running a similar setup to that a home (Using Windows Server 2003 on file server though), just run a quick test and I'm not having any problems playing four different 1080p 8GB MKVs on my desktop,another two on my laptop and two more on my HTPC simultaneously. I'd probably be able to do more but I'm out of CPU cores

So streaming should be fine with one NIC. What about when you are also copying an 8GB file to the NAS at the same time ?

If I can utilize both server nics without the need for subnets and bonding then can anyone suggest how to do it and a cheaper replacement for the HP Procurve that would fit the bill. I can get the Procurve for around 200 Quid (GBP)

One problem I have at the moment with my current setup is that if the nas is running a torrent client then the media player which connects wirelessly usually cannot connect to the network. The XBox also has the same issue. If I stop the torrent client (transmission) then I have no issues. The client is very rarely maxing the download speed of the internet connection but may be maxing the upload. This is why I am keen to put the backup NAS on the ADSL router and away from the main switch.

Thanks all
RB

 

[RimBlock]

Not actually running it myself, as I have a hardware RAID card (Dell Perc 5/i) and a Windows Server 2003 setup. I have heard good things about it though, but Openfiler might be a bit complicated to set up in a home environment (iirc it wants a proper domain setup). FreeNAS also supports ZFS and is easier to set up.

From a bit of reading it seems that ZFS is kneecapped in Linux but has a full implementation in BSD and Open Solaris. Whilst I work on Solaris 8&10 daily at work, I would rather not have it at home as I am much more used to Linux (RHE/CentOS) than Solaris from an SA point of view. Does look good though with RAIDZ2.

Really though you should schedule your backups for the early hours of the morning when there is almost no other activity.

Ideally yes but the uploads will be manually instigated. I guess I could 'tag' items to be backed up and have a nightly job copy the tagged files over only.

Even without bonding you can have two nics in a single PC on a subnet, just assign them different IP addresses.

Doh.. Yes of course I could. I blame old age for not realising that. All the traffic from the NAS will be in response and if it comes in on NIC-1 then all the responses will go back on the same NIC.

If you need expandability in the future, you just grab another 8 port switch and daisy chain it in. They are only around £20 a pop, much cheaper solution per port than getting a 24 port.

Yeah but still prefering the more robust 24port switch. Have had a procurve (8 port) in the past and had 0 problems with it.

Only thing I can think of is that your router is terrible and is maxing it's CPU/RAM out managing torrent connections, and thus isn't serving DHCP addresses properly. What happens if you reduce the number of torrent connections (which would probably boost your speeds too if your router is limiting you) and/or use static IP's?

Hmm, top of the range for Linksys home routers. Would be very sad if it really was that bad .

Liking the latest revision much more. For the NICs, as stated you don't need to have bonding to utilise both. The left-hand devices can simply get to the NAS via 192.168.1.1 and the right-hand via xxx.2 and despite sharing the same subnet, the traffic will travel via the correct NIC. Also note that you don't necessarily need a smart switch to utilise bonding. Haven't setup a Linux NAS yet but on the Win servers I have done, which had 2-4matching NICs, the manufacturer (e.g. HP/Dell/Intel) supplied a suitable driver and utility to enable them to emulate a single virtual NIC in the OS. The NICs can then be plugged into any standard unmanaged switch, so not reliant on LACP. It's possible your OS might offer NIC teaming natively. The reason I suggested keeping the smart switch is because they have better features, lifetime warranty and crucially build quality that you won't get from a cheapo £20 8-port.

Yes, I would agree and this is why I like the idea of geting the procurve.

The Netgear GS724T is on promo and can be had for ~£175 (with a free fibre module that you could maybe sell to recoup some value).

Interesting. Thanks, I will take a look.

The HDD/NIC layout in the NAS is the only thing slightly concerning me now. Not because of performance, more to do with setting up and managing everything e.g. setting up the right bindings and handling syncing with a drive that's significantly smaller than the main drives. If you're confident that you can handle it and the benefits are worth it then I guess you're good to go.

Uploads will go to the 500GB drive and then will be automatically distributed to the two 2GB disks on a nightly basis. The 500GB will just be for uploads, the OS and maybe DHCP.

I've seen this behaviour on some routers where an internal IP table (storing info on current connections) maxes out the available RAM forbidding more connections until others time out. I had this on a Zyxel HW660 in particular.

Again, that would be very sad if that was the case considering the price of the router. I have another 1Gbit switch I could use to move the NAS off the Linksys and I will see if that stops the issues. Thanks to you and Zarf for the info on that.

I'm liking the simplified layouts of the network design. It looks like you don't need the initial complexity you thought you would seeing as those with experience of multiple HD streams still can't max out disk or network connections. In which case the simpler the better, and cheaper!

Yep seems to be the case and what is better is with the 24port business grade switch I can modify the infrastructure quite easily as more requirements come up.


Thanks again all.

RB

 

[RimBlock]

Ok, moving on slightly.

I have bought a couple of Intel 1000/pro network cards, an Icydock 5 drive cage and an adaptec 1405 RAID card for a Raid 5 implementation (3 data & 1 parity). The last bay for the Icydock will be for the boot drive at this point.

I will populate the array with 1.5TB WD Caviar green drives (I have one already) and the boot drive will probably be my Hitachi XXXGB drive in my desktop machine as I have just got a SSD for it.

I also got another WD HD Live TV. I have the AC Ryan device but I find it is not so good with playback high bandwidth HD content. I seem to get lines like interlacing artefacts we used to see on badly encoded VHS copies. I read that the problem could be due to the Playon HD using the same chip for video as it uses for networking. The WD does have problems with high bandwidth HD as well but it will skip frames and the problems seem to not be so common. Shame as the Playon has a much better interface.

The HP switch prices are up to around 300quid now so I need to shop around a bit more to see if I can find it cheaper.

Cheers
RB

 

[RimBlock]

And it changes again... for the NAS.

The Adaptec 1405 is a HBA even though it was advertised on a large on-line store as being a RAID card . Will teach me for believing what the seller is saying.

I now have;
5x WD-CG 1.5TB drives in software Raid5.
350GB boot drive
250GB upload drive (possible to be sold)
Adaptec 2410SA PCI-X raid card (4 ports) - does not see drives over 1GB .

I am really not having any sort of luck with Raid controllers so I will stick with software Raid for now and use the 2410 with my Windows desktop and my Vertex II drives.

I have ordered a cable fro the Adaptec 1405 HBA as I have run out of ports (6) on the motherboard and the cable can be used for a better adapter if I choose to upgrade later. The HBA will be used for the boot / upload / non raid other share drive. The motherboard will have the 5 raid drives and the dvd rom drive.

Only 2-3 weeks to go before seriously looking at buying the networking stuff and starting an install.

RB