Need advice on storing lots of data - NAS/SAN

feenster99

feenster99

feenster99

Associate
Joined
19 Jun 2003
Posts
1,680
Location
West Yorks, UK
Hi all,
I am planning a project that will require Terrabytes of data storage. There will be a central server that does the normal server stuff, with as much storage as I can afford attached to it somehow.

So far, I have looked at using lots of NAS boxes, such as the Buffalo Terrastation Pro 1TB units. Assuming they cost £500 (ex VAT) per unit, and could handle 500GB in RAID 1 for data mirroring, is there a more cost effective way of doing it? I am guessing I will need to start at around 3 or 4 TB, but it will increase continuously, limited only be the amount of physical space available :)

Any advice appreciated.

Cheers,
Matt
 
The Thecus N5200 can do a 5 disk Raid 5 and give transfer speeds of 35mb/s and is about £600 inc vat but exluding HDD. With 5 Seagate 750GB HDD this will give you 3GB of storage using Raid 5.

Another option is to get a similar setup to my server - a CM stacker case with 2 exta cages gives space for 12 HDD, a Asus A8N premium with 8 SATA connections for two 4 disk arrays and a IDE boot drive.
 
if your planning on using lots of LAN boxes, the best and cheapest way would be to build a PC to do it.

that way you can have everything in one box, and just get yourself a hardware RAID 5 card, and re-use old bits to make the PC. doesnt have to be much ,just a celeron and 256mb of RAM or whatever will do

but spend the money on a proper Hardware RAID 5 card, and it will be a more convenient and cost effective way of doing it. but at the penalty of space (4 NAS boxes wont take up as much room as a full tower case for example)
 
Well, without revealing too much, there is likely to be racks full of these things in the future. It won't be viable to use lots and lots of PC's running OpenFiler for example. They need to be self contained units. Do the normal NAS boxes such as the Terrastation Pro's transfer data at such slow speeds then? The Thecus boxes seem to make a point of being known as the quickest to transfer - around 10x faster than other solutions.

Whilst we're on the subject, is RAID 5 suitable for storing lots of data and keeping it safe? I have been reading up on it, but my brain can't get around the fact that the data isn't mirrored - it is just re-buildable from the parity disk.

What other options are there except for NAS boxes or PC's filled with HD's?

Matt
 
Whilst we're on the subject, is RAID 5 suitable for storing lots of data and keeping it safe? I have been reading up on it, but my brain can't get around the fact that the data isn't mirrored - it is just re-buildable from the parity disk
Click to expand...

That's the way most large server databases are stored, RAID 5. Raid 1 is just too wasteful of space. If security is that important, add an archive backup device as well, eg tape or DVD store. The array provides the fast access to the data you need in the short term, the archive is for data you don't need instant access to.

One other way is to use a PC with SCSI card and have the drives in an external SCSI box, but SCSI drives are generally smaller than IDE or SATA one's and much more expensive.
 
malcolm said:
That's the way most large server databases are stored, RAID 5.
Click to expand...
Not strictly true, running a proper database on a single RAID5 array is a bad plan because it's effectively a single spindle. Normally when I'm speccing database hardware I use separate disks/arrays for the database and transaction logging, a single disk or array just can't cope with the I/O rates.

Better yet, do what I'm doing for my current baby - SAN storage with data on effectively RAID50, striped across 9 RAID5 arrays and logs striped across another 3. I've seen sustained transfers of over 300MB/s with the disks less than 25% busy. Beat that with Raptors :D
 
I am sure Matt will be very happy for us to get him to pay for an enterprise spec storage system :)

The cheapest way would be a cheap and chearful PC in a big case and an 8 (or 16) channel 4x PCI-E RAID 5 card (plus HDD!). Alternatively a couple of 5 disk NAS boxs will give up to 6GB on RAID 5 (approx £2k). If you have £2.5k and more, then a full SAN system with multiple redundencies is realistic.
 
Thanks for all the replies. Too many acronyms in the storage market!

Perhaps if I reveal slightly more of my plans, you might be able to better spec me. There will be a server running Linux, connected via Gigabit network cabling to the Router (probably a good spec Draytek). I want all the storage to link to this server, and allow users to access it via FTP. Basically, a very large capacity FTP server.

Now, nothing new there. However, I need it to be as cheap as possible, and infinitely expandable if possible. Hence, having a load of TerraStation Pro's on a shelf seemed ideal. However, 3 MB/s transfer rate doesn't sound too appealing.

Does that point me in a more obvious direction?

Cheers,
Matt
 
Hehe, yeah as I say, that will do me for up to 8/10 hard drives or so, but i'm probably going to be needing 30+ in the long run. Therefore, I am looking for some rack mount units or self contained boxes like the TerraStations.

Matt
 
feenster99 said:
But in the medium term, I will be needing perhaps 10TB or more, so that quickly won't be an option. I really need some self contained unit of sorts.

Matt
Click to expand...

You need a SAN, then you can grow the storage without having any impact on the service, no downtime whatsoever.

I deal with Hitachi Data Systems SAN....

4gb fibre connection, no single point failure, the SAN will phone home to Hitachi in the even of a disk failure, the first thing you know is Hitachi phoning you up to arrange on site time to put the new disk in, sustained throughput that will make your eyes water etc etc.

All the good stuff.

You are looking at many thousands of pounds though, over £20k. This is fairly high end stuff but storing 10tb is a fairly high end requirement.
 
Stolly, thanks for the reply. It definitely seems the right way to go in the longer term. In the short term however, I am going to have to stick to NAS boxes or similar, as we can't afford to invest £20K straight away.

Cheers,
Matt
 
I am going to have to stick to NAS boxes or similar
Click to expand...

it seems the TerraStations cannot sustain much more than 3MB/s apparently, and the Thecus ones can do around 30MB/s
Click to expand...

You are aware that once several users start accessing these files, the speeds above are going to drop? drop quite dramatically. We used to do tests on using SATA 200MB 16MB Cache harddrives for image storage, however once multiple users would start accessing it, there speeds would drop to less than 1MB per second (down from around 60MB+ sustained), unless the actual load on the storage is going to be very very light, then its not going to be worth using them, as your going to end up facing performance problems.

As somebody else has mentioned you're probably better off going with a SAN system, as if your business does grow at some point like you're predicting it will, you will end up having to move away from a consumer class NAS system to a business class NAS system and that could potentially mean a shed load of wasted money in old redundant hardware.
 
feenster99 said:
Stolly, thanks for the reply. It definitely seems the right way to go in the longer term. In the short term however, I am going to have to stick to NAS boxes or similar, as we can't afford to invest £20K straight away.

Cheers,
Matt
Click to expand...

It wouldn't be that much to start with, maybe half that. You'd expand it over time to end up with 10tb.

NAS is limited in terms of performance. It doesn't have much headroom, and doesn't scale (Well, it does if you by a high end NetApp box, but thats SAN pricing anyway) . Heck, i'd rather take a cheaper SATA only SAN and run it over ethernet than use NAS.

Thats just me though.
 
Hi,
I've been trying to have a nose around the 'net and find out what hardware I would need for the most basic of SAN systems, but am struggling a bit. Even Wikipedia has let me down.

Would you be able to give me a basic idea of what hardware I would need?

Matt
 
You must log in or register to reply here.
Back
Top Bottom