"Hardcore" NAS idea (figured better here)

Soldato
Joined
11 Jun 2003
Posts
5,251
Location
Sheffield, UK
I'm planning to take a small, low powered system, make it as quiet as possible and then dropping an infiniband/10gbe card into it with a straight cable connection back to my main case.

The box would be multi-homed with a regular gigabit networking card in it for the other pc + laptop etc to use as network storage. I plan however to mount some of the network paths on this box a local drives for my main games machine (as far as subst command in windows so they really DO look like local drives - this a good idea or keep is a regular locally mapped network drive?).

How feasible does it sound? Not really played much with higher end network kit but a pair of (probably crappy/old) infiniband cards on the bay are about £25 each and a short run of cable seems to be in the £40/50 range so it doesn't seem like a bad setup (probably have the other parts spare).

I can do the hosts file edits etc to get that side smoothed out (as the NAS would share the same network my gaming box uses to get internet access) and I'd probably use something like windows home server on the NAS end of things.

The reason for all this is to move my hardware raid 5 out of the main games box (where it gets a little warm, causes massive delay in booting the system and causes a fair bit of noise) to a network drive without loosing any speed from it. The raid 5 in my rig at the moment can happily do about 400MB/sec r/w so gigabit ethernet isn't really going to cut it to keep the speeds up. Using some form of fibre networking isn't really going to add MUCH noticeable delay (would be keeping SSD's etc local so I can live with 15ish ms access times between network and seek at the drive end) is it?

The idea spawned from this:

http://davidhunt.ie/wp/?p=232

So any additional info/tips would be appreciated :)

Edit:

There seems to be a fairly significant CPU hit for running SRP. Considering the gaming rig is a 2600k I could probably mess with processor affinity a bit on games run to leave 1 core to handle it?
 
Last edited:
Is there a reason why you'd do this instead of adding a bigger/second SSD to your gaming machine? You can then hide your new NAS well out of the way somewhere and never see or hear it.

Might not be a bad time to start a ZFS RAID-Z pool with some larger disks whilst you're at it.
 
You'd still have to deal with SMB overheads and the delays introduced with that. Honestly it sounds like you don't need the array at all since all you mention is gaming, but if you absolutely must have a bunch of disks stored out the way of your PC then I'd get a fairly simple drive enclosure with a SAS/SATA expander in and put an appropriate RAID card in your PC. You won't solve the boot time issue but how often do you really reboot?
 
Well, I've got rather used to being able to throw around fairly large files (be it dvd images or just games folders) between SSD and the raid with obviously fairly decent throughput. Moving to any form of regular NAS is going to rather cripple the speeds available.

The point basically came from thinking about NAS, having files available from any machine in the house without duplication or having to leave anything else switched on without loosing any speed.

As far as I'm aware SMB relates to the TCP/IP stack, I'd probably be using SRP (as mentioned in the edit in the link). I'm still not sure what sort of overheads it may have, I'm just putting an idea out at this point for holes to be poked in it.

As for noise etc, the NAS I could still "bury" fairly well. Even a pretty short cable run can go through a wall.

If there's any suitably cheap setups that could use any form of reasonably fast system to system interconnect AND allow the same drives to be seen from a network, that would be fine too.
 
To be honest if you don't need a TON of fast local speed storage then a regular NAS and more local SSD's will probably be better. However, it's fun to play with these things I'll grant and a good learning experience. I'll repeat what I said about starting fresh though. Get some higher cap drives and use an OS that support ZFS as your base.

I had this sort of setup in various guises for a time before SSD's became affordable. I don't really miss it to be honest, although I'll still go 10GbE when I can.
 
There is a certain amount of "mucking around with it" included here to be honest. Drive wise they aren't ideal for nas but I'm sat on a total of 4 1TB spinpoints. It's not cavernous storage but it's decent enough for my immediate needs. A shift to something like a RAID-Z pool is always a possible of course, I certainly have a few other disks I could throw at it (or flog these and go to cav reds or something).

As far a raid-z goes, I'm still getting my head around how you get performance above that of a single disk in the raid. Any good sources for how it all hangs together?
 
Last edited:
Well, I've got rather used to being able to throw around fairly large files (be it dvd images or just games folders) between SSD and the raid with obviously fairly decent throughput. Moving to any form of regular NAS is going to rather cripple the speeds available.

The point basically came from thinking about NAS, having files available from any machine in the house without duplication or having to leave anything else switched on without loosing any speed.

As far as I'm aware SMB relates to the TCP/IP stack, I'd probably be using SRP (as mentioned in the edit in the link). I'm still not sure what sort of overheads it may have, I'm just putting an idea out at this point for holes to be poked in it.

As for noise etc, the NAS I could still "bury" fairly well. Even a pretty short cable run can go through a wall.

If there's any suitably cheap setups that could use any form of reasonably fast system to system interconnect AND allow the same drives to be seen from a network, that would be fine too.

Well, technically, it would work if set up correctly, there's nothing wrong with it in that respect...I did similar for a little while connecting a micro-server to my workstation with 10GigE cards to test some things out until I could procure a 10Gig switch to use...

However I'd say it was a bad idea for most people, stuff like this it's usually a better idea to go through the temporary pain of changing the way you work a bit rather than concocting some unusually custom and usually somewhat fragile solution.

There are only a few specialised areas where large quantities of fast storage are anything more than a luxury (video editting etc) - assuming yours isn't one of them then for just throwing files around like you say, the answer is likely a decent NAS, gigabit ethernet and then leaving the copy running in the background (it's not sexy or clever but it's a very cheap and reliable solution).

Yeah, it's fun to play and push the envelope a little but you need to balance that with having stuff which works sometimes, if those files matter a bit to you (ie. you'd be upset if you were fiddling with something and lost them) then probably best not to build super custom setups like that because they will eventually go wrong one day and you won't always be able to fix it.
 
The idea spawned from this:

http://davidhunt.ie/wp/?p=232

So any additional info/tips would be appreciated :)

A few of us have already done this :

http://forums.overclockers.co.uk/showthread.php?t=18442377

I've had very good success with it. Built a smaller PC with a low power i5, 8GB RAM, HP P410 raid card in a Silverstone TJ08B-E case. I run Windows Home Server 2011 on an SSD drive and at the moment have 6x 1TB drives running in RAID 10. I've kept it fairly simple and just have a drive mapped from my main PC to the RAID drive. Any other devices just map/browse to it via the standard gigabit connection on the server.

Windows to Windows copies run at about 3.2Gbps which is maxxing out the RAID performance. I've also ran iperf between them and managed about 6.4Gbps using several streams just to see what they are really capable of.

Haven't noticed any performance bottleneck from the cards. But I only use it as a storage NAS and for running a few 24/7 apps.
 
Back
Top Bottom