We recently had a power outage and upon restore of power and start of ESXi (6.5) a number of VM's now show as inaccessible in vSphere Client:
When I SSH into the server and list the volumes I do not see that particular volume anywhere. All of the disks are active as per usual.
I am stuck at this point. Not quite sure where to go since I can't even find the volume for the particular VM's as listed in the vSphere Client.
it has been a few years since those VMs were set up, but I believe I have an idea which one they were on (BDGI3). I did a rescan from VSphere Client (RescalAll HBAs & RescanVMFS), but no change.
I did run a search in the /volumes folder for all *.vmx. I do not see any of the ones that show up as inaccesible there 
I used the find command in SSH to do this. It is odd to me that the volume that shows in the first image: 554cb8a-2f526e1f-acb0-60eb6920d734 is nowhere to be found.
**EDIT** No drives have gone missing during this outage, so I would expect to see that volume under vmfs/volumes
Here are the only messages I get in the vmkernel.log file after running that command:
Doesn't seem like a whole lot to go on
ISCSI for the 2 (Diskstation / ReadyNAS), the rest are internal drives on the server.
I pulled all of the drives (none showing error) and loaded DiskInternals Recovery tool. Then mounted them individually in an Insignia USB Disk Docking station (whatever it is called lol). Looks like one of the drives showed up properly and one did not. I think somehow the file system is corrupted because it shows like this:
The BDGI matches with the server. There should actually be a BDGI4 (which I had forgotten about). That one is now showing up as the one in yellow above. All other drives show up properly as BDGI1, BDGI3, and BDGISSD1 when mounted in the docking station.
So - I'm hoping this tool can recover something. But, are there other options?
well, I went a different route to try to recover data but still at a loss
I did the following:
- Loaded ubuntu on my laptop
- Installed vmfs-tools
- Hooked up the USB docker with the drive in it
- Terminal Commands to show the volume
- sudo fdisk -l (to find the disk, which was at /dev/sdb
- sudo fdisk -l /dev/sdb
- Terminal Commands to try to mount the drive:
- sudo vmfs-fuse /dev/sdb1 /mnt/vmfs
I initially tried just vmfs-fuse, but wasn't 100% sure it was a VMFS version 5 volume, so I tried 6. You can see that neither works And - when I try then, then the /dev/sdb1 seems to disappear.
I REALLY need to get the data off here, but it seems like ESXi corrupted something. Any other ideas?
well, VMFS Recovery by DiskInternals found all the files and folders. However - I didn't realize it would cost $1599 to license it and recover all my files
So - my next step was to try CloneZilla and attempt to clone the disk (minus the bad sectors). It is running, but not 100% convinced it will work. The reason is to make sure I don't accidentally ruin the original.
I set it up in advanced mode, set up rescue mode, and set it to NOT fix bad sectors. It is running now, Ran a pretty good clip, but now it is just scrolling through bad sector after bad sector for the last hour. Because, when this happens, PartClone (part or CloneZilla) seems to write all over the screen - it is difficult to read. From what I can see, I have about 70 hours remaining I guess I will just let it ride out.
My only other thought was to use dd_rescue to try to run a similar routine.