How do you test your backups?

[CraigN]

Im looking at a DR offsite atm and i was asked to provide info as to our current restore times.

At present i have a mixed setup of VMware and some physical pizzabox servers. The VMware is on a HP BL460c blades using an EVA 4400. The pizzaboxes are a mix of DL360s ranging from G3 to G6 and DL380's again from G3 to G6.

I have given a rough estimate of 3-4 weeks to restore from LTO3 (physical server backups) and LTO4 (VM server Backups). At present i backup all servers even VMs using the agent method.

The FD's face went white when i mentioned a month to restore just the critical servers. So im looking at some form of replication.

however i have now been asked to test this. Restoring servers from my Tapes.

Ive been thinking im gonna need some spare servers and another VMware environment along with another tape drive. So has anyone else done this if so how?

Ofcourse the lowest price solution will be best received by the man with the purse strings.

 

[CraigN]

that is just the thing ive never tested my backups so i dont know how long to restore.

We have a massive amount of data to restore about 52 TB on our san atm

That includes sourcing new servers yes all new hardware to our spare site recalling tapes rebuilding or restoring our infrastructure then individual servers and data.

Im looking at 72 servers in total. 3 Large SQL boxes and 2 massive file store servers that our document management system is on.

so my question is do you guys test your backups and if so how as i really want to get it done to give an accurate timescale.

 

[CraigN]

this is the other part of the issue

i was very conservative with my estimates as i want them to shell out for a replicated solution.

We were going for another C7000 with another EVA4400 and cisco ip distance gateways and san to san block level replication.

They put the stop to that after we did our main site due to cost. So im looking at an iscsi san and some large pizza boxes atm.

But i still need an accurate restore time as i am now i will have to buy new hardware and restore everything from tape, tapes that im not sure will work because ive never restored from them.

We have had quotes for a hosted DR solution but i used to work for a large outsourcing firm and i would prefer to keep it inhouse.

Could you give a bit more detail as to how you do your DR testing.

 

[CraigN]

Bigredshark and little crow very helpful thanks

We already have VRanger we purchased it when we went through our virtulization project however what we didn't scope for was the massive amount of storage we would need for Vranger to back up to before taking it to tape. I being new to VMware at the time did not have the experience needed for troubleshooting VCB based issues and backup exec is something i know. I have since got vranger working but i simply dont have the available storage to use it. I am actually in reading tomorrow at symantec looking at BE 2010 as now vsphere has got rid of VCB i want to see how they will do it going forward.

Regular check of my backups is what i want. My problem is I cannot restore my backups to the live environment, im currently looking at buying a pizzabox with a couple of quads and a load of ram to have a single test environment to test my restores on.

I also use LTO4s i do full backups friday/ weekend and diffs every day I send tapes off site to iron mountain everyday.

Bigredshark - i have done this we already have what we call a zero to four group (0-4) and then other groups after that based on business critical boxes first.

Our environment is strange we actually have over double that number of boxes. 72 at last count (inc VMs and Physicals).

Im currently looking at 3 options:

Rolls Royce Option: Replication of the current Blade + SAN environment in a second site with block level SAN replication. This is my preferable solution with very quick recovery should we loose a site. Downside is Massive cost more storage at our end new blades and SAN + expensive Cisco Fibre to IP distance gateways.

Second Option: 2 or 3 large Pizzabox servers With a iSCSI San. I have been looking at Left hand networks HP and also Dell kit. Would however require software replication Like Doubletake or something similar i have tested this to a degree, would also require me to up my WAN lines to cope. Roughly half the cost of option 1.

Third Option: Hosted solution, we are a small company and im hands on i don't like the idea of these things being taken away from my control so im all ready against this. It also comes in at £10k over Option 2, would also need a WAN line upgrade and if you invoke they charge you £10k per month of use.

All of these options would meet the criteria of having the business up and running within 4 hours of a total loss. Which is the aim.

But this has gone slightly off track I merely wondered how people test there backups be it in a test environment, on the live environment etc etc.

Thanks for all input

 

[CraigN]

This is uncanny

I currently Use ICM/Servo/Phoenix for our current hardware support. (whatever there called now, they keep changing names and im not sure who owns what)

The contracting arm of Servo was who we went to for our VM Supply and setup.

However there off site stuff seems very expensive. hmmm