ESXi issues after vCenter Server upgrade - cannot SSH and console appears different

[aj800]

We recently upgraded to vCenter Server 6.0.0 Update 2 in an effort to upgrade our environment to 6.x+ from 5.1.0.  After the vCenter upgrade completed fairly smoothly, I noticed one of the 2 hosts we have in the cluster had an alert - "vSphere HA agent is unreachable".  There is only one VM currently running on this host and I tried to migrate it to the other host so I could reboot the ESXi host to see if it would fix the issue but got a "general system error" and it failed.  I also tried the "Reconfigure for HA" option when you right-click the host in the vSphere Client.  In trying to further troubleshoot (check logs, etc.), I turned on SSH and the ESXi Shell but I could not SSH to the host... and when I connect via iLO to view the console of the ESXi host, it does not show the normal Black/yellow VMware ESXI standby screen, only a black screen that has at the top of the screen 'ESXi 5.1.0' (in yellow text) and the VMware URL (in blue text) with the copyright info on the next line.  After a while and hitting keys to wake it, it prompted me to login, but I'm not sure what I'm logging into or how to proceed further, or if a restart is necessary (or whether or not it can be rebooted via the client).

How can I fix this so I can move forward with upgrading the hosts to match vCenter?  The other host seems to be normal and encountered no issues like this.

[dbalcaraz]

Hello aj800,

You can't connect via SSH?

Did you check the ESXi firewall?

Could you try to connect to the host via C# client?

[aj800]

Yes, as mentioned, I updated the ESXi firewall via the C# client to allow SSH Client/Server and started the services and also for ESXi Shell.

[dbalcaraz]

Maybe you should restart the Management agents for the host in order to see if everything come up correctly.

More information: VMware Knowledge Base

[aj800 · Accepted solution]

So it was simply a matter of rebooting the host.  The issue was, there was a single remaining running VM on it that I could not migrate off to another host, so I was attempting to remedy the issue in that state.  Fortunately, we had a maintenance window where patches were applied to the VM guest OS, which required rebooting the VM, so while it was down, I was able to migrate the VM to another host.  When I completely cleared the ESXi host of VMs, I entered Maintenance Mode on it and initiated a reboot from the client.  It came back up normally, viewing it's progress from the console, loaded to the normal yellow/black ESXi console screen, and appeared in the client Hosts and Clusters view as in Maintenance Mode again.  When I initiated the exit of Maintenance Mode, the notification panel showed installation progress of the HA Agent, so I assume it had to reinstall the HA Agent again after the vCenter Server did not successfully install it the first time following the upgrade.  Once it was out of Maintenance Mode, and once the agent installation had finished successfully, the VMs were automatically migrated back, per DRS settings and affinity rules.