Guys, after a little advice please. What is the fastest possible storage for networked data access? We would need about 6TB worth so would it be setting up SSD's raided or is there anything faster available? Price point is pretty much immaterial to be honest. What kind if suggestions could you come up with? Cheers!
Help please. Networked data storage.
- Thread starter fwb2006
- Start date
Soldato
- Joined
- 7 Jul 2010
- Posts
- 3,581
YOUR BASKET
12 x Intel 520 Series 480GB 2.5" SATA-3 Solid State Hard Drive - Retail £639.95 (£7,679.40)
Total : £7,690.78 (includes shipping : £9.50).
12 x Intel 520 Series 480GB 2.5" SATA-3 Solid State Hard Drive - Retail £639.95 (£7,679.40)
Total : £7,690.78 (includes shipping : £9.50).
YOUR BASKET
12 x Intel 520 Series 480GB 2.5" SATA-3 Solid State Hard Drive - Retail £639.95 (£7,679.40)
Total : £7,690.78 (includes shipping : £9.50).
Would be rapid if set up right. OP what kind of network do you have? What kind of sustained throughput can it deliver?
Soldato
- Joined
- 7 Jul 2010
- Posts
- 3,581
Actually you would probably be bottlenecked by the RAID controller.
Man of Honour
- Joined
- 30 Jun 2005
- Posts
- 9,515
- Location
- London Town!
You don't need the fastest available, there are plenty of solutions available which can far exceed the data demands of several high end workstations. What performance do you actually require? What is the application? (or, when you say fastest do you mean throughput, IOPS, or access latency?).
My experience with high end storage is SSDs are rarely appropriate because they suffer have a limited lifetime and that can get hit pretty quickly. We have some seriously high end databases which would at a glance be the ideal use case for SSDs but we'd end up replacing them every 6 months actually. Large numbers of fast SAS drives tend to win in these scenarios. SSDs have a place as accelerators for caching if your application is suited to that approach (not all are).
If you actually mean throughput, which I suspect, then you need to consider your process, moving very large quantities of data over your network as a matter of routine is not generally a scalable approach. In any event, you'll need 10GigE to the desktop which will not be cheap, you will need to avoid cut through switching as it's inappropriate to large data transfers which will make it even more expensive (like, for a dozen workstations I wouldn't expect much change from £20k). Without that it's pointless thinking about storage as even a fairly small number of SAS drives can saturate a few 1Gig links.
I'd again caution against thinking that fast network storage is a solution, it rarely is, even at big media companies (who have shedloads of data to worry about) it's an approach which is avoided as a rule.
My experience with high end storage is SSDs are rarely appropriate because they suffer have a limited lifetime and that can get hit pretty quickly. We have some seriously high end databases which would at a glance be the ideal use case for SSDs but we'd end up replacing them every 6 months actually. Large numbers of fast SAS drives tend to win in these scenarios. SSDs have a place as accelerators for caching if your application is suited to that approach (not all are).
If you actually mean throughput, which I suspect, then you need to consider your process, moving very large quantities of data over your network as a matter of routine is not generally a scalable approach. In any event, you'll need 10GigE to the desktop which will not be cheap, you will need to avoid cut through switching as it's inappropriate to large data transfers which will make it even more expensive (like, for a dozen workstations I wouldn't expect much change from £20k). Without that it's pointless thinking about storage as even a fairly small number of SAS drives can saturate a few 1Gig links.
I'd again caution against thinking that fast network storage is a solution, it rarely is, even at big media companies (who have shedloads of data to worry about) it's an approach which is avoided as a rule.
Associate
- Joined
- 20 Oct 2009
- Posts
- 2,137
- Location
- Deepest, darkest Essex
My guess ( and only a guess ) is that what your really after is the fastest solution to share data, from lets face it a relatively small data pool, amongst a few workstations ?
By that I mean, cost no object ( talking a grand or so or far less ) standalone NAS or Server solution
Correct?
By that I mean, cost no object ( talking a grand or so or far less ) standalone NAS or Server solution
Correct?
Thanks for the reply guys. Basically we have a highend 3D application installed on a tasty HP server (256gb RAM and 32 CPU) which we want to share out to upto 20 concurrent users from a Windows Terminal Server via Citrix. The application is flat file and does not support multi threading yet. The application is very chatty with the database do we are thing of having the storage directly on the server to help speed things up. Previously we have/had the application installed local on the machines and accessed the projects over the (1gb) network but the performance of the application has been poor. The main reasons we the latency between the app and the networked project, poor data model and volumes of data we use. It is quite normal for a user to be working in data sized in the region of 20Gb so pulling all of that up and down really slows down performance. Also as it stands the application uses Sqlite as the method for controlling multi user access which is poor. We are looking at maybe moving to mysql but that is sometime off.
Hope that makes sense and look forward to more suggestions. Cheers
Hope that makes sense and look forward to more suggestions. Cheers
Man of Honour
- Joined
- 30 Jun 2005
- Posts
- 9,515
- Location
- London Town!
If it's just file access I'd look at any low end SAN/NAS packed with 300GB 15k SAS drives (performance at a reasonable price point). If money is really no object then consider Netapp as it has CIFS/NFS onboard if you license it, is reliable and with the appropriate disk setup is quite performant. If you can use a pure SAN then I have a lot of time for Hitachi (HDS) currently, the AMS2100 arrays are impressive bits of kit at a reasonable price and (again with the right disk setup) will saturate 8Gig FC with ease. Either of those will probably run to £30k or so with support and the like.
But be aware that move 20Gb of data around will not be quick under any standard, 8GB FC is the fastest storage generally available today and does 1600MB/s in theory. 4x wide SAS arrays comes close for significantly less money, theoretically they can get to 1200MB/s Only infiniband is faster today (much faster) but isn't a practical option on cost and complexity grounds.
It may be worth considering loading today's HP MSA of choice with 300GB SAS drives and connecting it by SAS, it will be the most bang for buck certainly.
But be aware that move 20Gb of data around will not be quick under any standard, 8GB FC is the fastest storage generally available today and does 1600MB/s in theory. 4x wide SAS arrays comes close for significantly less money, theoretically they can get to 1200MB/s Only infiniband is faster today (much faster) but isn't a practical option on cost and complexity grounds.
It may be worth considering loading today's HP MSA of choice with 300GB SAS drives and connecting it by SAS, it will be the most bang for buck certainly.
Soldato
- Joined
- 11 Oct 2008
- Posts
- 3,833
- Location
- London
SANs and Networked storage are useful for when you want to share your expensive disks with many servers, manage the storage pools easily and also scale all that managability across several boxes.
Unless you plan to virtualise more servers and services why not get Direct Attached Storage just for that server?
An HP D2700 with 2.5" disks (13 x 600GB 10k disk RAID6 + Hotspare to reach 6TB. Add more disks if you want more IO in different raid configurations (RAID10, RAID50)) and an HP Smart Array P812 RAID card would do the job.
This avoids having to set up all the redundant pathing and additional storage network components. Not to mention a similarly quick SAN or NAS (2 x 10GbE) would cost upwards of £15000 minimum.
I don't know the I/O requirements of your database software but once the sequential read and capacity needs are met, you are just chasing I/O performance with more spindles.
Unless you plan to virtualise more servers and services why not get Direct Attached Storage just for that server?
An HP D2700 with 2.5" disks (13 x 600GB 10k disk RAID6 + Hotspare to reach 6TB. Add more disks if you want more IO in different raid configurations (RAID10, RAID50)) and an HP Smart Array P812 RAID card would do the job.
This avoids having to set up all the redundant pathing and additional storage network components. Not to mention a similarly quick SAN or NAS (2 x 10GbE) would cost upwards of £15000 minimum.
I don't know the I/O requirements of your database software but once the sequential read and capacity needs are met, you are just chasing I/O performance with more spindles.