VMware Cloud Community
Ilikethat44
Contributor
Contributor

ESXI Host not responding to Vcenter after upgrade

I am wondering if anyone has seen a this before.

I upgraded my Vcenter to the Latest Version:  7.0.2.00200  Build number  17958471 

which went fine. Then i proceeded to use the update manager to bring my ESXI from 7.0U1 to VMware ESXi, 7.0.2, 17867351 and thats where my issues started> Some of my hosts updated ok some failed. So for the failed ones i mounted the CD remotely and updated
the Host from my ILO console. It all looked fine the first day but as i tried to do some backups via Veeam i got errors about problems accessing host. Also i start to see issue where Vcenter reported that quick stats for a host are not up todate. When i restart the network Management interface it doesn't get fixed.  Only fix is to reboot host and then it works for some time before it fails. 

Interesting part, the VM's work ok but i can shutdown any neither from the Vcenter nor the Local Gui of the host. Also if i try to tell the system from Console to restart via the F11 it shows restarting but it never does. The only way is a Hard reset from my ILO console.

So far the only fix i found is to completely reinstall the host with the fresh esxi install.

Wondering if anyone knows what causes this and if there is another way to fix this  

Reply
0 Kudos
8 Replies
IRIX201110141
Champion
Champion

Does your Hosts use SD/USB for its boot device?

Regards,
Joerg

Reply
0 Kudos
Ilikethat44
Contributor
Contributor

Yes all those are running / booting from Micro SD Cards which i have installed on mother Board. But they all also have a 250 gb SSD for Dump files and other stuff.

Reply
0 Kudos
sjesse
Leadership
Leadership

Reply
0 Kudos
Ilikethat44
Contributor
Contributor

Not sure as the errors described are not the same. Like i said before the VM's keep running fine but do no longer respond to shutdown from Vcenter and or the hosts web interface. It seems reinstalling has mad it a bit more stable but today i had error on one host that the SD card cant be read anymore and any changes will not be able to be saved.

In devices the card showed up with an error so i moved my vm's and rebooted and the host was able to still boot / read from SD card. So for now i am reinstall the software on a SSD Disk i have in it and will see if that removes the errors on this host.

Reply
0 Kudos
IRIX201110141
Champion
Champion

For sure youre affected by the same issue like countless other. Isnt there a sticky thread about the issue in the  vSphere ESXi forum? We see "too many tasks" in the  logs ... when we get gain access to the logs after waiting a long period for.

All 3rd. party Apps and vCenter overload the ESXi because of the boot device issued. All VMs which are up are running fine ... as long as you dont touch the VM.

What did you see of when try to log into ESXi trough SSH and try something like df -h or just opening vmkernel.log or hostd.log. If you reboot a host it will run a couple of days depending on how much action there is normaly going on (like Snaps for Backups ...)

Regards,
Joerg

Reply
0 Kudos
Ilikethat44
Contributor
Contributor

Honestly i did not try to telnet to the Box as after Vcenter could not reach it and the local web gui was also not ableto respond, nor telling the system to shutdown i always had to resort to the reboot from my ILO. But yes you are correct, it will run fine for a few days and then just take a crap , the VM's are up and running fine but all backups etc start to fail like Veeam as it no longer gets info from the Host for backup. I consider this a majr issue and this should be somewhere on the top of warning of VMWare that if you use SD cards or USB's that you might have major issues.

Just an update, i tried to move some VM's to SAN storage to update a host wehn they got stuck without ever copying the disk files at arround 29%, so i telnet to my host to run df -k and it never showed me anything... After some time it came back with this error

VmFileSystem: Slow refresh failed: Cannot open volume: /vmfs/volumes/5fb5a18b-dc3f4d10-f80c-38eaa78f5104

 

Reply
0 Kudos
Roger92
Contributor
Contributor

Hi,

Is there any news on this issue?
I'm having the same problem too, Veeam can't make backups anymore, VMs work fine but I can't shut them down.

 

Thanks

 

Reply
0 Kudos
IRIX201110141
Champion
Champion

@Roger92 

https://kb.vmware.com/s/article/83376

and avoid using USB/SD as ESXi boot media in the future because its marked as deprecated now.

Reply
0 Kudos