Time Machine size issues with VMware Fusion 2.0

[essd]

All,

I've been using VMware Fusion since 1.1 (and VMware on the Windows for much longer), and I've been enjoying the support for Time Machine backups since the feature was released in Fusion 1.1.2.

Since upgrading to Fusion 2.0 this past week, the amount of data backed up by TM has exploded. In the past, if I booted a VM, it might have backed up a couple of segments of the virtual HD (maybe 2-4 gig) after using the machine. Now, since upgrading to VMware 2.0, it almost always seems to back up the full 15 gig image every time. My disk is set up in <= 2 gig segments and I don't understand why the whole image should be copied every time.

A listing of the VM directory shows the following timestamps on the disk files:

-rw-------@ 1 staff 1633222656 Sep 21 19:53 Windows XP Professional-s001.vmdk

-rw-------@ 1 staff 1920794624 Sep 21 20:04 Windows XP Professional-s002.vmdk

-rw-------@ 1 staff 1062273024 Sep 21 19:53 Windows XP Professional-s003.vmdk

-rw-------@ 1 staff 2146041856 Sep 21 19:52 Windows XP Professional-s004.vmdk

-rw-------@ 1 staff 498335744 Sep 21 19:53 Windows XP Professional-s005.vmdk

-rw-------@ 1 staff 1745092608 Sep 21 19:52 Windows XP Professional-s006.vmdk

-rw-------@ 1 staff 1974337536 Sep 21 19:52 Windows XP Professional-s007.vmdk

-rw-------@ 1 staff 1964834816 Sep 21 15:29 Windows XP Professional-s008.vmdk

-rw-------@ 1 staff 1866137600 Sep 21 15:29 Windows XP Professional-s009.vmdk

-rw-------@ 1 staff 877133824 Sep 21 19:53 Windows XP Professional-s010.vmdk

I only use this particular VM for: (a) editing a small number of tiny Microsoft Word documents, and (b) running Microsoft Money. There is certainly nothing running that should be touching every section of the hard drive (and even if there were, this behavior suddenly started happening only after upgrading to Fusion 2.0).

Does anyone have any suggestions? My TM backup is to a remote server (to which I am connected wirelessly), so this 15-gig-backup-every-day thing is killing me.

[admin]

Time Machine operates at a file, not block, level. If just ten bits were changed (spread out over your entire virtual disk), Time Machine would have to back the entire thing. You may have been lucky up till now and the 2.0 upgrade changed the balance of what got touched.

The potentially large space requirements is one reason I don't suggest using Time Machine with a virtual machine. Instead, I'd suggest less frequent manual backups, backing up inidividual files in the guest, or keeping your data on the host and accessing them through HGFS shared folders or network shares.

[WoodyZ]

While technically one can certainly backup only segments that changed at one give moment this does not make for a valid backup that can restore the target Virtual Machine at a later point in time and without getting into technical details let me just say that to backup anything less then the entire Virtual Machine Package while the target Virtual Machine is not running and or the Virtual Hard Disks not mounted via VMDKMounter and preferably with Fusion closed then one does not have a complete and valid backup of the target Virtual Machine Package from which to effect a complete recovery to the point in time is was backed up.

To broach just one of the technical aspects of why I'm making the previous statement is that with split disk's and snapshots the various vdisk segments and snapshots are chained to one another and to only back up pieces of a virtual hard drive in any respect and not the entire VMP and expect to maintain an unbroken chain especially over time is unlikely to happen. Although some broken chains can be repaired under certain circumstances the breaks that could/would occur by doing a Differential or Incremental Backup do not really fall into that category.

Personally I do not nor would I use Time Machine to backup my Virtual Machines nor do I backup the VMP that often however I do backup User Data off of the Virtual Machine to appropriate storage media on a regular basics and do have Recovery Images of the Virtual Machines so I can quickly recovery when necessary however to backup the entire VMP over and over on a regular basis is just not necessary and is a huge waste of space and time.

[essd]

All,

I understand the intricacies of trying to take backups of stripes of VM disks (I've debugged filesystem drivers before), and I certainly would not rely on a TM backup of a VM that was currently in use at the time of backup. I do, however, believe that it is reasonable to ask Time Machine to take a backup when the VM is suspended (or powered off) and have the backup work properly and without copying stripes that have not been changed.

For that matter, I have isolated at least one behavior of Fusion 2.0 that is causing the issue that I noted: it seems to update the timestamp of each and every disk file whenever you suspend a machine. I admit that I have not done extensive trial and error, but just after I pressed the "suspend" button on my machine just now, I noticed that all of the timestamps were bumped.

Regardless of the wisdom of backing up VM files with TM, I believe that the above item is a bug in Fusion 2.0. Can anyone else confirm this behavior?

- Essd

[WoodyZ]

I do, however, believe that it is reasonable to ask Time Machine to take a backup when the VM is suspended (or powered off) and have the backup work properly and without copying stripes that have not been changed.

Okay, you believe what you want to however doing a Differential or Incremental Backup of a split disk and or snapshots will lead to a corrupt and unrecoverable virtual hard drive when restoring from less then full backups so when you cannot restore a virtual machine because you've been doing Differential or Incremental Backups don't say you weren't warned!

[rhind]

For that matter, I have isolated at least one behavior of Fusion 2.0 that is causing the issue that I noted: it seems to update the timestamp of each and every disk file whenever you suspend a machine. I admit that I have not done extensive trial and error, but just after I pressed the "suspend" button on my machine just now, I noticed that all of the timestamps were bumped.

I can't confirm the behaviour (haven't checked as my current VM is a single file as was created with 1.1 but I plan to convert it). But if you open a file for read/write but don't write to the file, when you close it, does the modified time get updated? If so perhaps it is that Fusion 2 keep all chunks open for quicker access whereas maybe before 1.x only opened chunks when they were first needed?

If so, then perhaps there is no way round this.

Cheers

Russell

[essd]

WoodyZ,

I admit that I do not actually store any user data on my VM, and that I would probably be a little more paranoid if I did. As it stands, replacing one of the VMs would be a big pain due to the need to reinstall and reconfigure everything, but it would not result in any data loss, so I can afford to be (slightly) more trusting.

Having said that, I still find it hard to believe that VMware would officially enable support for Time Machine backups (as they did in the 1.1 series) if there were no way to make it work.

There is absolutely no technical reason why one cannot do an incremental backup of a split disk so long as the machine is turned off when at least one incremental is run. Yes, all of the backup segments need to be in an internally-consistent state within the Time Machine snapshot that one wishes to restore, and one cannot expect to make any sense of a backup done while the machine was turned on. However, restoring from a powered-down snapshot is an entirely reasonable request for any backup software.

Remember that Time Machine is smart enough to recognize when files get modified in the middle of a backup operation. Even if Time Machine runs while the VM is in operation, meaning that a segment or two may be in an inconsistent state for that Time Machine snapshot, the disk segment's mtime will be updated whenever a write occurs, so the inconsistent segments will be replaced by consistent segments during the next Time Machine operation when the VM is shut down.

I would use the word "broken" to describe any behavior that does not fit the preceding.

As for whether or not closing the disk file would cause the file's time to be updated...that is a good question, but even if it did, there is also no reason why Fusion couldn't keep track of whether or not the disk was written to, and then touch the file date to reset it back to whatever it was before. I will go see if I can file a bug on the issue and see what they say.

- essd