vData Recovery deleted the vmdk files that I didn't "tick" to backup

[sportsystems]

We have recently migrated to ESXi 4.0 from ESX 3.5, and the upgrade went smoothly. We decided to adopt vDR as initial testing has been positive, it has been backing up several of our VM's without any problems, until last night when I set it to backup only the C: drive and H: drive VMDK's of a multi-disk server, it seems to have gotten itself confused and in the process has deleted the unticked "data" disks completely - without trace.

Our VM (called "fs1") has 5 disks which were allocated accordingly (each disk in it's own datastore+LUN on the SAN)...

C: 20GB (vmstore1/fs1.vmdk) OK

E: = 450GB (filestore1/fs1file1.vmdk) deleted!

F: = 500GB (libstore1/fs1lib1.vmdk) deleted!

G: = 500GB (libstore2/fs1lib2.vmdk) deleted!

H: = 500GB (libstore3/fs1.vmdk) OK

As you can see, the two that are named the same were untouched, but the 3 that were not ticked got wiped out!

I went onto the "unsupported" console to verify with my own eyes as the datastore browser can give varying results... none of the extra disks from this particular VM were there... the folder structure was there, but the vmdk and -flat files were missing.

I know this sounds crazy, but it's the only coincidental thing I can find that may have been the cause - that and the two disks happen to be named the same (which is why they were both ticked in the vdr Backup screen as it wasn't obvious which disk was which).

I've yet to create a support ticket as I'm busy recreating the data stores and doing a restore. If I get chance I'll try to recreate this in a dev lab, or if someone else could I would be greatful.

Edit just to say, the only errors that appeared in the events was this by the vDR backup user around the time the VM went offline (due to losing its disks), an error -3941 failed to create snapshot:

"Create virtual machine snapshot

fs1

File <unspecified filename> is larger than the maximum size

supported by datastore '<unspecified datastore>"

[athlon_crazy]

wow, that's madness!

vcbMC-1.0.6 Beta

vcbMC-1.0.7 Lite

[sportsystems]

I know, I cannot explain it.

All I know is, the VM was powered off overnight as a result of losing several disks, not sure if that was vSphere that powered it off or Win 2k3 decided to shutdown - nothing in the event logs other than an unexpected shutdown event.

All 3 affected datastores had 100% free space when I looked, and the only 2 that were still intact were the ones that I had ticked to backup (because I wasn't sure which one was the C: drive). I can only think it's related to unticking those "data" vmdk's and/or possibly related to the snapshot error.

[sportsystems]

Just to update this, VMware Tech have called and basically advised to switch to VCB and wait until the next "proper" product release of vDR 1.1 which is due either late this year or early 2010.

[alubel]

Whoah, that really sucks.. backup product deleted the data it was backing up..

This makes me extra nervous because if the company that makes the product doesnt feel confident in it, then whats the customer supposed to feel?

The Vstorage Api is good, just the VDR isnt?

[sportsystems]

Yes it seems the vStorage API is fine, but the logic/internals of the vDR appliance has severe flaws, certainly enough for them to advise not to use vDR and to stick with VCB until the new version is released.

Think I'll leave it a good 6 months post-release before I try that product again. 😐

[ben13]

not sure if this is the cause but stumbled on this KB article while looking for something else.

Title :Snapshot Engine inadvertently deletes independent persistent disks in some cases

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1013929&sl...

thats a huge bug!

[sportsystems]

Good spot! That looks to be the correct KB article for our scenario, specifically this was relevant to us..

• A non-independent disk resides on another datastore that is formatted to a larger block size as the disk is larger than 256GB

...the disks that got wiped out were all 500GB on seperate datstores. :smileyshocked:

[sportsystems]

That looks to be the correct KB article for our scenario, specifically this was relevant to us..

• A non-independent disk resides on another datastore that is formatted to a larger block size as the disk is larger than 256GB

...the disks that got wiped out were all 500GB on seperate datstores. :smileyshocked:

Just had a thought, it still only partly explains it, because the 5th disk in this VM was kept intact (the disk named the same as the C: drive).