We recently suffered a power outage that lasted for longer than our UPS was able to supply.
This caused an unexpected power-off at an ESX host that houses two virtual machines - a FreeBSD that has a ZFS pool for storage purposes over NFS
And a second virtual machine - a Dev Windows Server, whose VMWare files (including the .vmdk) were located on a datastore, which was in fact part of the ZFS pool, and was accessible over iscsi/NFS.
After the outage, the Dev server remained inaccessible until the FreeBDS was brought up, which was to be expected.
However, once the NFS datastore was accessible, and I tried powering on the second machine, I got an error message that the .vmdk file was missing!
I've tried several things thus far - running diagnostics on the ZFS pool seemed to show no problems with it, but I'm currently running a scrub on it, just in case...
The file is simply "not there" - i.e. not accessible either through NFS, or locally, within the shared mount on the ZFS - and the "Used space" does not account for a file of that size, either...I am at a total loss, and would greatly appreciate any help regarding a repair/recovery procedure anyone may suggest.
Thanks in advance!
SSH to the host that has access to the datastore where the VMDK was located and "ls" the VM folder ie:
ls- alh /vmfs/volumes/datastore-name/vm-name
Does that output show a vmName-flat.vmdk file?
If you see vmName-flat.vmdk but are missing a vmName.vmdk then your VMDK descriptor file is lost, however your data is still there.
You can recreate the missing descriptor file with this KB Recreating a missing virtual machine disk descriptor file (1002511) | VMware KB
Let us know how you get on.
Unfortunately, no such files. As a matter of fact, no large files at all.
Some piece of additional info, which I omitted previously, but may or may not be, of importance.
After the FreeBSD ZFS machine was brought up, the host kept on showing the Win Dev VM as "inaccessible". So I proceeded to remove it from inventory, I browsed the datastore to the .vmx file and select "Add to Inventory". It was successfully added, and when I tried to power it on, upon reaching 35%, the host brought up an error message, asking if I moved, copied, or did smth else with the machine...I selected that I moved it, and from that point on, the only error I've been getting, is that the .vmdk file is missing...
Also, there are several vmware-xxx.log files, one of which was huge, prior to my failed attempt...now they are all showing a size of 46.3K.
Is there a way, with a VMWare or linux tool, to scan the folder on a lower-level, and check if the filesystem still has the data on it? I am certain that the power outage did something with the NFS/iSCSI connection that "broke" the file, as it was being accessed at that point...
Btw, the scrubbing of the ZFS passed, and reported:
scrub repaired 0 in 5h36m with 0 errors on Thu Nov 24 01:24:38 2016
So, the ZFS is intact.
Finding lost vmdks is completely different with iSCSI and NFS.
Please tell us which of both you actually use.