VMware Cloud Community
HendersonD
Hot Shot
Hot Shot

VCB in hot-add mode, why are backups slow?

All of my VMs run in an NFS datastore on my Netapp filer so I cannot use VCB in SAN mode. Instead, I just implemented the new Commvault virtual server agent in hot-add mode. The virtual server agent job backs up 22 VMs which took just over 7 hours to backup 11.6GB of data on an incremental backup. Why so long for only 11.6GB of data?

When I look at my VM logs this is the type of entries I see when a VM is being backed up via the virtual agent:

Create Virtual Machine Snapshot 12:00:39 AM

Create Linked Clone 12:02:19 AM

Create Linked Clone 12:02:37 AM

Remove Snapshot 1:22:43 AM

I have 4 VMs being backed up at once with the whole process kicking off at midnight. My guess is the first 4 VMs have to go through the entire process above before the next 4 kick off. This means the first 4 did not complete until the snapshot removal process close to 1:30AM. An hour and a half for each set of 4 VMs would jive closely with a total 7 hour backup time.

Why is this taking so long?

Reply
0 Kudos
6 Replies
AndreTheGiant
Immortal
Immortal

If you have a NFS backend why you do not backup directly the files (after a snapshot)?

Are you using a FileLevel backup mode instead of FullVM?

Andre

Andrew | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
Reply
0 Kudos
HendersonD
Hot Shot
Hot Shot

The terminology here gets a bit confusing. VCB documentation talks about an image level or file level backup. The Commvault virtual server agent has three choices:

Disk level

Volume level

File level

In addition, if you choose disk or volume level, you can check a box to gather metadata so even with a disk or volume level backup you can do file level restores. I have choosen a disk level backup which I think corresponds to what VCB calls an image level backup. I am gathering metadata inforrmation so on a restore I can restore and entire VM or individual files in a VM. Using this virtual server agent my fulls on weekends are much larger than my incrementals during the week. Something in this whole process is taking a good bit of time and I cannot figure out what it is. I do know that a snapshot of the VM is taken, then a cloning operation, and then finally the snapshot is eventually removed. The snapshot happens very quickly but after that I am not sure exactly what is happening in the backup process. Is this cloning operation (whatever that is) taking a long time? Is the file scan that must occur in order to figure out what has changed taking a long time? Is the actual copying of the changed files from the VM to my disk based backup target taking a long time?

If I understood hot-add mode better, I may be able to explain this

Reply
0 Kudos
AndreTheGiant
Immortal
Immortal

Is this cloning operation (whatever that is) taking a long time?

Yes, if you use image level backup (I suppose disk level in Commvault)

If you use file level backup, VCB will only create the snapshot and mount the snapshot (instead of make a copy of the entire disk).

Andre

Andrew | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
Reply
0 Kudos
HendersonD
Hot Shot
Hot Shot

So I want to make sure I have this correct.

If I choose image level backup (what Commvault calls disk level), a snapshot is taken, the entire snapshot is copied to the VCB server, and then this cloned copy is scanned for any changes. Whatever has changed will be copied to my backup destination.

If I choose file level backup, the snap is taken and the scan happens on the snapshot. Whatever is changed is then copied to my backup destination.

In other words, in hot-add mode a complete copy of the VM has to be made first before the copy of changed items is made to my backup destination. This is what is slowing things down. Is this correct?

If it is, if I switch to volume or file level backup then that should speed things up

Reply
0 Kudos
Rumple
Virtuoso
Virtuoso

Just to put in 2 cents that arn't really required, you really should be just using snapmanager for VI and letting the filer do all the work of integrating with VC to put VM's into snapshot mode and then creating filer snapshots. You can then mount any of the snapshots and copy files out of the VM if required (or just powerup the VM while its in the snapshot and use storage vmotion to move the vm back into production). Recovery and restores of VM's become a non issue then...

Using VBC with Filer nfs is not the best solution to be using.

Reply
0 Kudos
HendersonD
Hot Shot
Hot Shot

Rumple,

Thanks for the response, I was thinking about doing exactly that. If I do purchase the license for SMVI, does the process happen in a timely fashion. If I remember correctly, first the VM is quiesced, a VM snapshot is taken, a filer snaphsot is then taken, and finally the VM snapshot is deleted. I own snapmiror already so is SMVI integrated into snapmirror?

Reply
0 Kudos