rsync on esxi4 server

[CNBarnes]

For various reasons, we back up the datastore of the VMWare ESXi 4 server using rsync (rsnapshot).   This was working when we ran 4.0 (after much work to get it compiled), but since we upgraded to 4.1, it no longer works.

And now we have having a devil of a time trying to get rsync working on the esxi server.     Has anyone had success with this?  Even better, is there an FAQ document that shows how to do this?

[DSTAVERT]

Not supported at all but http://www.virtuallyghetto.com/2011/02/how-to-compile-statically-linked-rsync.html

[drummer925]

I've looked at that page before. Unfortunately those directions only work with ESXi 4.0 which I discovered. I tried staticly compiling the latest version of rsync on my debian machine and then scping it to the server running ESXi 4.1, only to get a warm segmentation fault. I realize ESXi is a proprietary OS, but would know how I could replicate the ESXi environment for the sake of compiling code for it.

[lamw]

Did you follow the directions to the T? I only as ask you can't use the latest OS version of choice to compile the package as that will most likely give you a seg fault, I'm pretty sure it should work on both 4.0 or 4.1 as the version of Busybox is the same

[drummer925]

Ok, I got it to statically compile for 4.1 and work. Rsync now runs and pulls backups from server running ESXi. However, I am pulling .vmdk files which many times give Device or resource busy errors. These vm's are running so I understand the problem with backuping them up. Maybe it would be possible to queue these devices and have them run during periods of no activity. Any ideas on this problem?

[lamw]

That's expected, you're trying to access a VMDK that's currently locked due to it being powered on, this has nothing to do with rsync. You need to take a snapshot and then the main VMDK will be in RO mode for you to replicate OR have power off the VM

[drummer925]

Given current configurations, these machines cannot be shutdown to allow for this to happen. Running a lsof confirmed these .vmdk were indeed in use and could not be access. Snapshots are not an option in a production environment.

[Dave_Mishchenko]

Is there a reason you don't want to use snapshots?

[CNBarnes]

Probably because snapshots are not backups.....

[lamw]

@CNBarnes, you're right snapshots aren't backups ... but that is not what is being asked here.

The OP wanted to rsync VM's VMDK, if it's powered on, a lock will be placed on it's disk making it impossible to copy the VMDK remotely whether you're talking about VMware's native vmkftools or some other utility like cp or rsync. Since it's not possible to shutdown the VM, one would need to create a snapshot which then creates a new delta disk for the writes and  that puts the main VMDK into read-only mode and then you can leverage  rsync else you would need to look into 3rd party solutions to leverage CBT (Change Block Tracking) which does not use snapshots and provides ability to backup live running VMs such as VMware vDR, Veeam Backup & Replication, PHD, etc. who leverage this new functionality from vSphere

[CNBarnes]

Guess again - the OP works for me.   :smileydevil:

We ARE trying to backup the entire VM.   The "normal" vmware tools are not available to us since we don't have multiple esxi servers nor run vCenter (vCenter being a prerequiste of pretty much any useful tool).   That means we need to use typical linux tools such as rsync & rsnapshot...

[lamw]

Well as I mentioned, you won't be able to acces the VMDK since it's powered on and locked ... "a" potential work around is a snapshot, but if you're not able to take one or take down the VM, the options are pretty slimm for backing up entire VM without going to guest level backups.

I haven't personally used Veeam or PHD's solution, so I don't know if it requires vCenter ... I know vDR does. So perhaps you might still be able to leverage CBT to backup the entire VM but I can't speak to vCenter requirement for those two vendors

[Josh26]

If your concern is that snapshots aren't real backups - rsync is no closer.

Have you looked at ghettovcb? At least with it, the solution may not be officially VMware supported - but all the technologies used are, and there's a large community of users.

http://communities.vmware.com/docs/DOC-8760

[CNBarnes]

Non-sense. 

We use rsync to backup 10 other servers, both a local backup as well as offsite.  This is a daily backup w/ 7 daily retentions as well as 2 monthly retentions.

[DSTAVERT]

@CNBarnes

You seem to have missed the point of snapshots. Snapshots simply freeze writes to the base disk and unlock it for reading allowing the rsync process to access the disk. Because all disk writes, (changes) are written to the snapshot delta file the base disk is in a point in time condition allowing rsysnc to calculate the differences. If the base disk were not frozen rsync would not be able to make those calculations since the vmdk would always be changing. Kinda like being interrupted in the middle of counting things, ya gotta  start over.

[CNBarnes]

The vm's are servers that are in production.  They cannot be stopped - at all - ever.

Even if a snapshot would be appropriate (and they are not), the fact that they freeze the vm means they cannot be used.

[lamw]

I don't think you really understand how VMware snapshots work, I would recommend asking one of your staff members to show you how it actually works in a development environment while you have a workload running. The guestOS and/or application is not "frozen" when it's taken and there's also an option to quiesce the OS such as VSS for Windows.

The main point to get across is to perform any type of "copy/replication" of the VM's disk while it's running, the main VMDK(s) must not be locked, a snapshot would re-direct all the writes to a new "delta" disk when the snapshot is created. This then allows you READ-ONLY access to the disk to copy off using traditional VMware tools such as vmkfstools or rsync, once that process is completed, the "delta" disk will then be merged back into the main VMDK and the snapshot is then removed. Snapshots are not backups and I think you understand that pretty well, but to get around the requirements you have set forth, it's the only solution other than the one I had described earlier in which you can leverage commercial products to see if it'll run with VMware vCenter in which snapshots are not used at all and a new concept of Change Block Tracking is utilized.