Back up question

anything I don't mind · 20 Dec 2011 at 09:41

Due to a problem that a previous engineer created by incorrectly assigning drive letters to SCSI drives, the backup and DR procedure has not been working correctly. When I first arrived i could not run vsphere and connect to vcenter because the vcenter installation was on one of the drives that had changed drive letter. They were using ca arcserve 12.5 to do backups. The backup software was not creating the vmdk files with the vcb proxy, it was unable to mount them.

I have created a new vm and install vcenter on it and upgraded the ca arcserve to version 16. It is now backing up vmdk files during the backup process and everything is working ok.

The problem now is that the backup is taking far too long. When I come in, in the morning the backup is still running after 12-13 hours. It is still running and the portable HD of 1tb still has space.

The question is, do you think it is overkill to backup files from the servers as well as vmdk snapshots of the vmware images. On this network there are only VMs and one physical server that requires backing up. Do I have any other option but to disable the backup of the files on the VMs and just use vmdk backups? any thoughts ?

Ev0 · 20 Dec 2011 at 09:57

I guess it depends on the business and how the backups 'fit in' with things.

I'm a relative idiot when it comes to the nitty gritty of backups, but I guess it comes down to time to recover something.

By backing up the files as well does it make restoring something quicker than if you had to mount up a VM and then pull the files off? What are the businesses expectations when it comes to backups/restores?

Yamahahahahaha · 20 Dec 2011 at 10:11

I agree with Ev0's points, especially the business expectations.

As for the backup job itself, which part of the job is taking the longest?
Doing the snapshots of VMDKs (relatively fast, I would think?) or backing up all the files?

anything I don't mind · 20 Dec 2011 at 10:13

Well in an ideal world I would have the file backups and the vm backups on one disk if it fits, but if a) it might not fit b) takes too long. Then I have to make a decision i guess.

The business expectation has never been specified officially. They just assume that everything is backed up and ready to be recovered at any time and that there is a full DR recovery plan from the backups as well.

The physical server and its SCSI drives from the san do host the DMS and a few other such systems, if there ever was an individual file recovery request, i would most likely be from there. Which is not a VM so it will be backed up by the file system itself.

What I am concerned about is wtihin the CA software there is an agent for exchange and an agent for sql. these appear as additional checkboxes under the server names within the backup software for exchange stores and sql databases. If i just back up the VM for those servers then I might be missing out on "special" exchange or sql backups. I might contact CA their support and discuss with them.

But as i don't have much choice i am probably going to take off all the file system backups apart from the physical servers and the exchange stores and sql.

anything I don't mind · 20 Dec 2011 at 10:18

Yamahahahahaha said:
I agree with Ev0's points, especially the business expectations.

As for the backup job itself, which part of the job is taking the longest?
Doing the snapshots of VMDKs (relatively fast, I would think?) or backing up all the files?

Just looking through the logs now and comparing, but from first glance the snapshots go at around 30mbyte a second and can take from 20-120 mins each depending on size.

Little_Crow · 20 Dec 2011 at 10:56

I take a vmdk image of the C drive for every server, I have also moved the paging file to a separate disk and excluded it from any backups.

Then it's really situational for the data drives. For critical DB systems then both a vmdk backup and SQL agent backup are taken.
Otherwise it'll be a vmdk backup or agent backup, but never both.

Without knowing a lot more about your infrastructure it'd be hard to give specific advice.

We use vRanger for our backups to disk, then they are shipped off to tape - might be worth giving the trial a shot and see if that runs any quicker for you.

Yamahahahahaha · 20 Dec 2011 at 11:07

groen said:
The business expectation has never been specified officially. They just assume that everything is backed up and ready to be recovered at any time and that there is a full DR recovery plan from the backups as well.

For your own sake, get one written down and get it agreed. When **** hits the fan, management will look for a scapegoat.
Also document times taken to restore things from the backup (this means you must test the restores too).
You are using a tape drive too, correct?

groen said:
What I am concerned about is wtihin the CA software there is an agent for exchange and an agent for sql. these appear as additional checkboxes under the server names within the backup software for exchange stores and sql databases. If i just back up the VM for those servers then I might be missing out on "special" exchange or sql backups. I might contact CA their support and discuss with them.

The Exchange Agent will do Database level backups and, with a bit of pushing, Document level backups too. The database level ones are fast but do not allow for single-item retrieval. The Document level ones rely on MAPI, and are slow as a result.
I can't think of what SQL settings there are off the top of my head.

groen said:
Just looking through the logs now and comparing, but from first glance the snapshots go at around 30mbyte a second and can take from 20-120 mins each depending on size.

FWIW, a client's R310 box running r15 SP1 to LTO4, which backs up Windows files, manages to do 20MB/s. 30MB/s doesn't sound unreasonable to me. Of course this depends on every link in the chain. If you're not happy with it, maybe look at D2D2T?

anything I don't mind · 20 Dec 2011 at 11:28

There was a DR procedure already in place when I arrived, that was last tested at the end of 2009. As the vmdk backup was not functioning for several months before I arrived the DR procedure was no longer available. We have a DR test in february and I am trying to optimise the process before then by improving the speed and however else I can. The client just wants the original procedure to be used and does not want many changes too it. After the test he wants me to do a report on how we can improve it. That seems backwards imo, but that is what he wants.

We have a Teralyte 2000-TLYTE-2RSA that backs up to 1 and 2tb SATA disks. http://www.idealstor.com/teralyte.php

We have a support license with CA that allows for updates to the latest version, but i am not sure if this D2d is included in that, will look in to it.

The biggest issue with this DR and backup is that the network is not on the internet and there is no p2p replication either. Due to security reasons. So i have to physically take the backup hds to the DR site a few hours away and then recover the VMs in to the DR environment.

Ev0 · 20 Dec 2011 at 11:59

Also remember that a file restore would not (normally anyway!) invoke the full on DR procedure

As has been said you need to get accepted backup and restore processess formally documented and agreed by everyone so that when the poop does hit the fan you cna at least be covered.

Yamahahahahaha · 20 Dec 2011 at 14:22

Arcserve includes staging (short term D2D2T) but won't let you backup to disk then push that backup to tape.
I will caveat this by saying that it will probably cost you a limb and your firstborn to get this working, even with Arcserve support on the phone.

From the client's perspective, they have a working policy which is under maintenance. Getting the backups working is the most important thing, optimisation is secondary.
I hadn't seen those Teralyte boxes before, but I'd still trust my DR data on a tape more than any SATA hard drive.