Help - all flat.vmdk files showing as -1 kb

[Zqz]

Hello all,

I am fearing the worst but thought I would ask the question any way.  We had a power outage last night and the UPS did not shutdown the our virtual machines (it's a long story). 

When the power came back on, I tried to restart the virtual machines but I get the same error for each one  saying

Failed to power on VM
Could not power on VM : Invalid metadata.
Failed to initialize swap file '/vmfs/volumes/xxxxxxxx': Invalid metadata

When I look at the flat.vmdk files, it says that they are -1 kb in size and modified on the 31st December 1969.

Has anyone ever seen this before and more importantly, can I recover my machines without resorting to a backup?

TIA

[a_p_ · Accepted solution]

Welcome to the Community,

I've never seen anything like this. I'd strongly recommend you immediately open a call with VMware, if there's important data on these VMs that cannot be restored from the backup..

André

[Zqz]

Hi Andre,

Thank you for the welcome - I was thinking the same but was hoping that someone might have seen it before and save me having to do hours of restores.

** update ** After calling VMware support, I am still baffled as to why it happened and so were they.  Also told me that they had seen it a few times with customers but they do not offer data recovery services.   I had to delete my datastores and reload everything from backups. $1200 for out of hours support well spent(!)

[continuum]

Do you still have the VMFS datastore with the corrupt flat files ?

If yes - please boot the ESXi with this CD
http://sanbarrow.com/files/moa-esxi-rescue.iso

and check if it also sees the flat files as 1 kb only

[Zqz]

Thank you for the response.  I had to delete the datastore as I needed the space.  The VMware support guy logged in and took a look too, he said that they were definetley gone, and that he has seen it a few times with a number of customers.   What was worrying (and still is worrying) is that he could not explain why it happened.

He said that it has become so common that VMware have written the following KB article  http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101541... - I had already paid for out-of-hours support and this sounded quite expensive.  I only called VMware to find out why it happened and to see if there was a quick way to recover, I guess I could have waited until business hours.

Luckily, I do an hourly backup to an external host  and was able to just delete the store and start again, so I only lost 20 to 30 minutes worth of work.