Thanks for the replies everyone.
What sort of data is this??
Student work, mostly video, audio and image files.
I would recommend you purchase an LTO-5 Autoloader (48 port). Each tape holds 1.5TB (compressed). This would give you 72TB of data backup storage.
I couldn't find a UK price for a HP/Dell autoloader but if we convert US $ prices, you're talking £10-15K.
It might seem a lot but nothing in comparison to loosing all company data.
That isn't actually as much as I had imagined. This may be an option I need to look into further. Thanks for the suggestion.
/edit The amount you need to backup does seem a lot. Is the 50GB the size of the SAN, or the amount of data you need to store?
It is the size of the SAN (Which has yet to be implemented yet btw), but I expect it to fill up very quickly, then a significant amount of the data to be flushed each year to make way for new data. So it should be running at about 70-80% capacity most of the time.
50TB of data? Important data? How much is to lose (in money) if this were to disappear over night?
Money wise it is unlikely to cost little if anything at all. It would however cause major problems as it is going to be used for HD video footage for students. So if we lost it all, a whole load of technical extenuating circumstances forms would be filled and a lot of peoples heads would be on the block. It would also not do good things for my heart (I am user support, but will also be the Admin for the edit facility as central IT want me off their network (Political reasons and stupid policies.) )
What kind of Mickey Mouse company is this? If you can afford to buy a fibre channel SAN and load it with 50TB, you can afford to take proper backups. As redundant as SANs are, they're not completely infallible. Also, what would happen in case of fire? If this is even slightly important data, which judging by the expense it will be, then you need to be taking offsite backups.
The most cost effective solution will depend on what kind of data this is and how many servers it will be presented to etc, but generally you're going to be looking at enterprise level backup software (CommVault or suchlike) with an LTO 4 or 5 library. Sure, this will end up being a 5 figure layout, but anything less and you're just asking for trouble in event of a disaster.
I would strongly recommend you sell this to management in terms of your BCP (you do have one, right?) and put quite bluntly that in case of fire, without the backup solution, you're completely dead in the water.
It is for a large university. As mentioned above, central IT want me off the network for political and policy reasons. The facility will consist of 38 Mac Pros for HD video editing/ Image editing etc. The whole project has lacked any sort of project management from the start, and I have taken it upon myself (probably stupidly) to get it sorted as I will be end user support and admin (Yet am not in a position to make major purchasing decisions etc.. sigh).
BCP? I wish! Well actually there probably is one somewhere that won't have been looked at or updated for years. Unfortunately this is well above me, although I am more than willing to push the issue, especially with regards to the backup as this would have direct impact on my if it went wrong.
The trouble is that if you are putting in a ~50TB capacity SAN then how to back that up should have been factored into the design from the beginning. It's not just a case of throwing in a LTO# auto-loader, media (not cheap when you consider how many tapes you would need for a reasonable cycle) and backup software licenses.
You also need to consider how service using the storage is going to use it and what limitations that its going to have on the backup.
Fortunately the SAN hasn't been implemented yet. The tender document is currently being written/finalized. So the sooner I can push the issue of backup, the better as I may be able to convince the powers that be to include it in the tender document. At the moment they are trying to do it all on cost, so think they can shave off money by not having backup! You can all see this is wrong, I can see this is wrong, but I just need some options and ideas so I can present them to and convince management.
What sort of time window will you have to back up the data?
- however good your tape drives are they will take time to back things up, (I have inherited systems before where the "nightly" backup ran for >24 hours!)
The 'downtimes' will be about 18:00-07:00, weekends, half term & holidays.
What type of data is it (e.g. is it flat file or Oracle database files) ?
- if it's Oracle data for instance you may need to backup using RMAN to allow for the service not to be taken down for the period of the backup ... this can add complexity (and normally licensing costs)
Mainly video files (ProRes Quicktime), Audio Files and Image Files, with a few misc files in there as well.
What is the rate of change of the data?
Changes will occur at least every hour, every day of the working week. It will however go in cycles of heavy use (A lesson going on etc), right down to really minimal use.
- Can you get away with a periodic full backup and then much smaller regular incremental backups ... but how often you do the full backup will depend on the answers to the following questions
Incremental backup would be fine, and how I initially imagined it.
What is the needed recovery time if a single file is lost?
There shouldn't really need to be a need to recover single files, unless they were really important ones. If that was the case then I would say <48hours? The main need for backup would be incase of a total failure of the SAN.
- Which backup (full or incremental) holds the latest copy of the file and how will you know this information?
As above, this may not be necessary. Any suggestions to this would be welcome though.
What is the needed recovery time if the whole lot is lost?
- Recovery of the full backup and then rolling though incrementals will take time.
This would depend on the time of year. But a week should be acceptable.
How will you test that the backups are working, (initially and ongoing) ?
- As others have said this is very important ... if you cannot prove that your backups are recoverable then you have no backups.
This is something I am not sure about, so again suggestions would be greatly welcome.
The most complicated backup solution I was involved in was a multi-terabyte Lotus Domino cluster which needed that all Domino database files be recovered to an arbitrary point in time either individually or as part of a full recovery of the service. This had to be a consideration from the start, (as the requirements for the configuration to be able to do this affected the hardware requirements). Turned out to be a quite complex solution but worked well but it certainly wasn't cheap but both types of recoveries were proven which made the Customer happy, (until someone else took over the support and screwed up the backup software completely ... but that's another story).
