Cheapest way to back up ~50TB?

I really cant see the point of the install here of a 50Tb SAN. If you are saying that all the students have local storage to work on, and that you are thinking of making it their issue to secure their data in the event of a failure, what is the point in a centrally managed storage soloution, which wont be managed properly.

With that amount of data, either it gets managed properly with scheduled backup windows and proper management buy in, or dont do it at all.
 
Sorry to butt in .. i'm new here.

Will the students be storing and working off directly on the SAN or will they be working on their local disk and then they would need to do the backup themselves.

If they will be working directly on the SAN, with the students reading/writing large HD files - are you sure that the SAN will cope? That is from a bandwidth perspective and also from the NAS perspective ( that is the OS firmware on the NAS ).
 
I'm inclined to agree with the later comments. As nice as it would be to provide a solution for them, I don't think it would be cost effective.

Perhaps offer them guidance and support on backing up their work, as well as finding appropriate tools for doing so?
I know it sounds like you're holding their hands a lot, but some people just aren't tech savvy.

The problem with this is it would cause more problems than it would solve. I would end up spending much of my time helping people back up work etc (A lot of user may not be tech savy at all, so I need to make it as fool proof as possible.. Also if we encouraged students to all get external hard-drives, they are likely to start working off them. This is ok for audio and still imagery, but if they did it with video work then it is going to cause me a world of grief. A combination of different makes, models, connections and file systems would again lead to my time being taken up with fault finding. I don't mind students backing up for their own peace of mind or to recover work whic htye themselves have messed up, but relying on them in the case of a major failure may not be the best thing.

I know people talk about fire / flood damage but at the end of the day if either happened I'm sure there would be a lot more at stake than some student videos gone missing, such as the uni / school being gone...

I can't see how you'd ever get it backed up if the files are changing that often unless you replicate constantly...

It's not just major fire or floods that would be the issue though. Water damage (And with the amount of luck I gave had with leaks and water damage recently, this is one that worries me), electrical problems, virus etc may all lead to a major data loss without wiping out the whole school.

The backup wouldn't need to be kept 100% up to date, but being able to recover say 1 week back should be sufficient and would save a lot of work.

I really cant see the point of the install here of a 50Tb SAN. If you are saying that all the students have local storage to work on, and that you are thinking of making it their issue to secure their data in the event of a failure, what is the point in a centrally managed storage soloution, which wont be managed properly.

With that amount of data, either it gets managed properly with scheduled backup windows and proper management buy in, or dont do it at all.

The students won't have local storage to work on. All of the students need to be able to access any of the computers and carry on their work. The options I see for this are; External hardrives, and as I mentioned above if the students provide their own this will cause more problems than it is worth due to the mix of different makes, models, connections and file systems. So we could provide them, but take 1000 students and all give them all a drive and you have just spend as much as or more than a decent SAN. Or implement a SAN, which will be managed in terms of user quotas, and hopefully backed up if I get my way.

Sorry to butt in .. i'm new here.

Will the students be storing and working off directly on the SAN or will they be working on their local disk and then they would need to do the backup themselves.

If they will be working directly on the SAN, with the students reading/writing large HD files - are you sure that the SAN will cope? That is from a bandwidth perspective and also from the NAS perspective ( that is the OS firmware on the NAS ).

They will be working directly off the SAN. Yeah the bandwidth and access times etc have all been carefully calculated and the SAN will be more than able to cope. We are aiming to get enough bandwidth from the SAN to be able to saturate Gigabit Ethernet for 26 machines at the same time. This is enough to provide 5 streams of Pro-res 422 for each machine at the same time.
 
They will be working directly off the SAN. Yeah the bandwidth and access times etc have all been carefully calculated and the SAN will be more than able to cope. We are aiming to get enough bandwidth from the SAN to be able to saturate Gigabit Ethernet for 26 machines at the same time. This is enough to provide 5 streams of Pro-res 422 for each machine at the same time.

What is the make of the SAN if I may ask?

I also read some comments re the dis-advantages of having the students working locally and then backing up to the SAN.

I agree with you that having a centralized place is wayy better and more manageable then giving all the students an external drive.

Maybe a bit of a silly question - but don't Macs include the Apple Time Machine backup? It's fully automated and and keeps incrementals (AFAIK). Why don't you just configure that for them as a set and forget.

I have a similar situation to yours ( albeit much much smaller :) ). Have a much smaller NAS (Drobo FS) and have the users working locally but then backing up directly to it. I am a PC house so I cannot use Apple's Time Machine but I am using the PC equivalent called Altaro Oops!Backup (http://www.altaro.com). It's fully automated CDP, includes ReverseDelta technology to store just changes and I also use it's backup maintenance policies (like backup retention policies) - that is I can configure to only keep a certain amount of version backups (based on size or time frame). A bit of silly name but works wonders.

I don't know if Apple Time Machine includes all these features as I never really had direct experience with it but from what I heard it's a great backup tool.
 
From previous experience (not in an education env though)...

If your at tender stage, get a price in for the king daddy solution.
2nd identical SAN
Relevant software to replicate (Continuous Access in HP's case)
Links to other site/building/floor/room of 2nd SAN
All the other junk you need to go with it
also include the tape library & software & server (would you not need to keep historical archives at all???)

and if the management decide to implement half a solution... tell them not to bother at all & give every student a USB hard disk or something lol

in seriousness though, have a plan for cutting out the 2nd SAN & utilizing just tape backup at worst, but make the very very aware of the consequences
 
that's a lot of storage !

no no really it's not, not even close, I'm sure some people here manage 100 times that amount, I setup a DR site for the office of national statistics and they had four 12 drive Sony Petasites, each one backed up 1494TB, they used a funky ILM tiered solution too so I dread to think how much data was actually housed between their six sites!
 
Dammm that's expensive, we managed to get one for a bit more than half of that. :D

I don't think we spent much more than that for a SATA Beast II fully loaded with 2TB disks....all 84TB. Recently added the expansion unit with another 40 disks to give us 164TB :cool: and still room for another 20 disks! :eek:

Disk to disk replication is definately the way to go....daily backups to tape are an ongoing nightmare and more and more of a legacy as storage gets cheaper.
 
Disk to disk replication is definately the way to go....daily backups to tape are an ongoing nightmare and more and more of a legacy as storage gets cheaper.

A client of mine has "just" 3TB of production data [videos, images, project files etc] and that takes a small age and a crap load of ballache to archive to tape. I'd rather buy a second SAN than arse about backing up 50TB of stuff to tape!

If this is student work, the onus should be entirely on them to ensure their work is safe. I am pretty sure my college and Uni both did the same - any data that goes missing and not backed up is your fault.

Having a 50TB SAN should be all the safety you provide tbh. As has been said, USB HDDs are dirt cheap these days. It wouldn't surprise me if a lot of students had them anyway - I know I would want to be able to take my stuff away to work on at home.
 
Personally I'd be investigating getting a big fat dedicated line installed and use an online backup solution. Initial setup will take time but subsequent backups can be on an incremental basis.
 
If you stipulate from the start that this data isnt going to be backed up, and users dump stuff on it at their own risk, then you're covered. We have a simular setup here, students have 30TB of dump space on our lefthand and they know its not safe. One day their dept might decide to buy another cluster of nodes to mirror their data, but what with the current climate, I doubt that will happen.

If you want cheap storage, the QNAP 16TB (raw) boxes are only about 3.5K each, you could potentially get 10 of those and have them mirrored for under 50K budget. They offer pretty good performance and features if all you want is file storage.
 
Snapmirror that suckah :)

What he said.
With that much data, your talking SAN - to - SAN replication or just don't bother.
Trying to backup that much data to tape will take an ice age, not to mention even thinking about restores.

As others have said, I'd place the load squarely on the students in that case.
 
Personally I'd be investigating getting a big fat dedicated line installed and use an online backup solution. Initial setup will take time but subsequent backups can be on an incremental basis.

I don't think a university will be lacking bandwidth! They're normally on multiple gbit links already. I imagine the Janet network is 10gbit + by now?
 
Back
Top Bottom