VMware Cloud Community
EdOfTheMountai2
Contributor
Contributor

ESXi 6 VMs "inaccessible" after New Years's power failure ?

My ESXi host power must have failed over the holidays. 

  • The VMs that were running over the holidays are grayed-out and display as "( inaccessible )".
  • The *.vmx and related files are still there.
  • The VM that was not running at time of power failure starts fine. 
  • My ESXi host has a single 2TB disk of local storage. 

I can browse and locate the vmx files using the vSphere Datastore Browser.  I tried removing from inventory.  However browsing to and right-mouse clicking the *.vmx data storage displays a disabled (grayed-out) "Add to Inventory".

I also tried ssh to the server and reloading without success:

vim-cmd vmsvc/getallvms

vim-cmd vmsvc/reload <invalidVmNumber>

Ideas and suggestions are much appreciated.

Thanks in advance,

-Ed

ESXi 6.0.0 Build 3029758

I am a ESXi non-expert user.


esxi-inaccesible.png

Message was edited by: Ed Sutton Added vSphere screenshot

Reply
0 Kudos
24 Replies
virtualosa
Contributor
Contributor

Maybe this will help.... https://www.veeam.com/virtual-machine-backup-solution-free.html

Though I haven't used it and don't know the limitations, but it should be better than nothing.

Good luck!

Reply
0 Kudos
continuum
Immortal
Immortal

Aborted by VMFS via a virt-reset on the device

2016-01-04T21:46:36.255Z cpu0:34502)WARNING: HBX: 4837: Replay of journal <FB 1506800> on vol 'hdd-2tb' failed: I/O error

....

Found this post late ....Do you still listen ?
If you have to run ESXi in small environments you must not have power problems !!!
With ESXi 5.5 and later an unexpected power dropout can be a complete dataloss.It is maybe a good idea to get some practice in troubleshooting - if the running ESXi cant read his own files it does not mean its unreadable.Let me know if there is still something to do.





________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
EdOfTheMountai2
Contributor
Contributor

>With ESXi 5.5 and later an unexpected power dropout can be a complete dataloss.


Thank you *very* much for the warning that ESXi 5.5 and later can destroy data on an unexpected power  loss.

Why would VMware not fix this problem if ESXi 5.4 did not have this issue?  It is  a journalling file system right?

I should have stayed with ESXi 5.4.  I never lost data using ESXi 5.4 after several years of 24 / 7 use.  After only a month of ESXi 6 I lost all my running VMs which were corrupted to the point that I could not recover them. 

>it does not mean its unreadable.Let me know if there is still something to do.


I ended up deleting my datastore , recreated it, and have started re-creating all the VMs on ESXi.  This time I will use VM Explorer to backup all VMs from a powered-down state to my NAS.


The VM files that were in use at time of power failure reported input/ouput error when I attempted to read them?  What tools could have fixed these files?


UPS power management will be my next priority.  I have a single ESXi host, a small Synology NAS, and a network switch connected to a APC 1500VA UPS.  The Synology NAS gets notification when UPS is on battery.  I am trying to figure out how to use NUT.  Until I get this tested and working I better start shutting down ESXi at night.

Thanks for the warning!

-Ed

Reply
0 Kudos
EdOfTheMountai2
Contributor
Contributor

Crap.  It happened again.

esxi-inaccesible-vm.png

I am kicking myself.  I was puzzled why UPS did not save it this time.  I discovered I had plugged it into the wrong power-strip. That move is going to cost me a lot of time.

Why is it so easy for a power failure to corrupt ESXi 6 files system and why is it impossible to recover from ESXi inaccessible file errors?

I don't recall every having corrupted a Linux physical machine like this before.

-Ed

Reply
0 Kudos
continuum
Immortal
Immortal

If you are lucky only the partitiontable is lost. This can be fixed quite easily.
There are a few things that you can do to prevent serious problems after power-failures - if you are interested call me - I can show you what to do.
And dont compare VMFS with normal Linux-filesystems - VMFS keeps a large part of its meta-data in RAM so unexpected power-failures will produce problems that do not occur with filesystems like NTFS, ext3 or reiserFS


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos