ESX 3.5 Backup Suggestions / Help

[EllettIT]

Hello All! It's been a while since I've visited these forums however recently I've been trying to get a plan together to back up our virtual environment. First, here's a quick description of our virtual environment:

3 Dell PowerEdge 2950's with 32GB RAM each, 6 NIC's (2 onboard, 2 dual port cards), dual Xeon 5345 processors each

1 Dell / Equallogic PS400E SAN

ESX 3.5 UPD 1 Standard x 3, VC 2.5 Foundation (not virtual)

Here's our current backup environment / method:

CA Arcserv 12 backing up each VM (local drive's + system state) as a "file level" backup, something in the neighborhood of 2 TB's - not a full VM backup

What I proposed we do was purchase the VMware Agent's for CA's Arcserv software and a disk based backup device (ExaGrid 3TB appliance) which would be installed and configured by our SAN vendor. The idea was to begin to do full VM Backup's alongside our normal backup process as a precaution for SAN firmware updates, semi DR, ESX software upgrades / updates, etc. Basically to give me peace of mind and a way to back out of what I would consider a "disaster" I.E. losing the SAN or a firmware update gone bad. Due to the cost of the solution the idea was shot down. Now I'm back at the beginning trying to find a "cheaper" alternative to giving me some peace of mind.

Some of the ideas I've had would be to purchase a cheap NAS with enough capacity to allow me to get a full VM backup prior to any changes I might need to make however I'm not sure as to what all I would need to be able to do something like this. Would I need the Arcserv agent's for VMware or something like it (maybe EsXpress or Vranger Pro etc) or could I get by with just VCB? Would it be possible to not even use VCB and mount the volume's on the SAN and copy the files over or would I need something like VCB to put the files in the correct state? Any thoughts would be appreciated, thanks!

[RParker]

Would I need the Arcserv agent's for VMware or something like it (maybe EsXpress or Vranger Pro etc) or could I get by with just VCB?

It's even less complicated than that, install VCB, point your acserve server to VCB. Done. VCB does the work, arcserve just puts those vmdk files someplace, all that's really taking place is a rar or zip of those vmdk files, vcb takes them offline tempoarily using snapshots, that in essence is really all that's going on. The VCB framework is making the vmdk 'visible' to your backup software.

[Chuck8773]

I don't back my VCB images up to tape yet but I can tell you what I know about VCB.

We have a VCB server. It has VCB installed. It is also connected to each volume on our EqualLogic SAN. Note: Do not initialize those disks if you open Computer Management.

I then have a number of command line scripts set to a schedule to back the VM's up to disk. VCB contacts Virtual Center to create the snapshot of the VM. VCB uses its connection to the SAN volumes to copy the data of the SAN. Then VCB tells VC to remove the snapshot.

Keep an eye on those snapshots. Sometimes VCB does not clean up after itself very well.

I do not know how this integrates with a backup software, but I hear it is very nice.

Also Note: I read once that you should NOT connect to these volumes using MPIO.

[EllettIT]

@ RParker

When VCB puts those files offline is the VM offline the duration of the backup? With Arcserv (or VCB) putting the files somewhere I'm thinking it wouldn't be wise to put them back on the SAN so maybe local storage (DAS) or network storage (NAS) perhaps? Or are you saying Acrserv would need to be able to grab the files from the SAN to be able to bring them to tape (or whatever)?

@Chuck8773

Is your VCB server seperate or on the same box as your Virtual Center install? It would make sense to me to put them on the same box since it's already connected to the SAN.

[RParker · Accepted solution]

When VCB puts those files offline is the VM offline the duration of the backup?

No. The snapshot feature was designed so you don't have to take the VM's offline.

With Arcserv (or VCB) putting the files somewhere I'm thinking it wouldn't be wise to put them back on the SAN so maybe local storage (DAS) or network storage (NAS) perhaps?

Yes, thaat's what we do as well. We don't occupy valuable SAN high performance real estate for archives, that would be a waste, so we utilize SATA local storage.

Or are you saying Acrserv would need to be able to grab the files from the SAN to be able to bring them to tape (or whatever)?

Probably better to stage it, save it to local disk first, maybe a NAS or Windows file store. Then use your backup software to treat those files like you would a physical server, and move those to tape. You can probably go direct to tape, but if the backup fails, you are wasting tape / time, by staging it it will be easier to manage also.

[fooldall1]

Sounds like you already have all the elements you need to make VCB backups a reality...... Like RParker suggested, create a staging area on your backup server (SAN storage, even on ATA disk would be more expensive) on a set of local disks and make that your staging area. Sounds like you already have a VCB proxy server set up- and you were wise to DISKPART AUTOMOUNT DISABLE and DISKPART AUTOMOUNT SCRUB on that VCB proxy prior to giving the Win2k3 box visibility to those LUNs. That can be a lot of fun to troubleshoot. Also, most modern backup software includes some type of VCB snapin, All you need to do is decide WHO is going to be the backup proxy (the preferred way is to make the backup server itself the VCB proxy, if possible) and what method to use..NBD, SAN, HotAdd, etc. In your case with the Equalogix SAN (assuming it's not iSCSI) you'll be using the SAN method. Sure, all this can be run manually, no backup software required, but wouldn't you want to be able to manage your backups and restores from a single place that everyone is familiar with? Not to mention having the ability to life-cycle the backups..

When you use the SAN method with your backup software, the backup server calls on the VCB proxy to issue snapshot requests to virtual center, or directly to the specified host. When that happens, the snapshot files are created that allow the VM to continue to function as usual- as long as the I/O isn't excessive enough that the volsync driver in vmtools can't keep up (SQL, Exchange). The base disk is then copied over the SAN to the local disk staging area on the VCB proxy. When that process is complete, the snapshot is removed by the consolidated helper as it commits the changes in I/O during that process to the base disk and removes the snapshot file once all I/O has been committed. That's the tricky part- in most cases, the process is almost instant, or may take a few seconds to a few minutes depending on the delta I/O captured during the snapshot process. Once the disk is in the staging area it's written to either tape, or disk- depending on the target of your backup job. GRT (granular restoration tech.) is pretty cool- since the current 1.5 version of VCB only does full backups of the vmdk each time- it gives you the option of just restoring individual files within a VM (or redirected elsewhere) instead of having to restore the entire vmdk, mounting it, and then pulling out a file...a real time saver, especially when the vmdk's backed up are large. VCB compact is pretty handy as well if you're going to tape- it just backs up data and no "whitespace". There are concerns there, of course if you're going B2D, because disk de-duplicating devices don't like compression.

I know a lot of this changes with vSpehere, but this is based on 3.5 u2.