ESXi 5.5 Disconnected Host

[pvevers]

Hello

Does anyone have any previous instances of a host routinely disconnecting in vSphere . The host itself is still up and can be accessed directly, however under the vCenter server that the hosts are being managed by it shows as disconnected. It can only be reconnected by removing and re-adding the host to the list of hosts managed by the vCenter server, or by restarting the host via iLO.

This is not the same issue as that which can be worked around by changing the network adapter to VMXNET3 from E1000.

Regards

Paul.

[UmeshAhuja]

Hi, Go through this below link... Will help you in analyzing your problem. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=205405...

[pvevers]

Many Thanks for this.

The log entries in that article are not present in our logs. Is this issue referenced in the article known to affect ESXi 5.5 as well?

I have found a KB article to walk through the next time it disconnects which may diagnose the underlying issue.

Many Thanks for your help.

Regards

Paul.

[UmeshAhuja]

Hi, Can you post some logs or error you are getting while host is getting disconnect from vCenter

[pvevers]

There is no error that appeared to be present.

I have retrieved the logs, the host is now connected. I can upload copies of vpxa/vpxd logs, or copy and paste the sections from when the host was last disconnected.

[sajal1]

Hello pvevers,

please provide the logs as well. Is the vCenter hosted inside a VM in the same environment?

[pvevers]

Hello

I attach the vpxd and vpxa log files.

I can confirm that this is the case with regards to the vCenter being hosted within a VM in the same environment.

Many Thanks

[sajal1]

Hello pvevers,

I could see the following error lines in vpxd logs

2014-03-15T09:20:30.108Z [05676 error 'SoapAdapter.HTTPService'] Failed to read request; stream: <io_obj p:0x0000000003c61178, h:2140, <TCP '[::1]:8085'>, <TCP '[::1]:55184'>>, error: class Vmacore::SystemException(An established connection was aborted by the software in your host machine)

This is a repeated msg in the file

Can you please check the following?

VMware KB: VMware VirtualCenter Server service fails to start when vCenter Server is installed on a ...

This may be a problem with vCenter. But can you also attach hostd logs of the host that is getting disconnected

• /var/log/hostd.log:
  
  
• /var/log/hostd-probe.log:

I am assuming that only one host is behaving this way or this is randomly for any host or all host?

[pvevers]

Hello

I attach the requested logs.

I can confirm that on the vCenter server involved that vCenter Server is installed on the C: drive.

Additionally, it is the case that it is only 1 host behaving in this manner.

Regards

[sajal1]

Hello pvevers,

I could see the errors :

2014-03-14T22:14:33.107Z [04604 info 'vpxdvpxdHostCnx' opID=SWI-7b803ee9] [VpxdHostCnx] No heartbeats received from host 52532b19-7dbe-7dec-2e15-f9f6db0bc5de within 89336000 microseconds

2014-03-14T22:14:33.126Z [06932 info 'vpxdvpxdInvtHostCnx'] [VpxdInvtHost] Got lost connection callback for host-2401

2014-03-14T22:14:33.127Z [02876 info 'commonvpxLro'] [VpxLRO] -- BEGIN task-internal-3528 -- host-2401 -- VpxdInvtHostSyncHostLRO.Synchronize --

2014-03-14T22:14:33.127Z [02876 warning 'vpxdvpxdInvtHostCnx'] [VpxdInvtHostSyncHostLRO] Connection not alive for host host-2401

2014-03-14T22:14:33.127Z [02876 warning 'vpxdvpxdInvtHostCnx'] [VpxdInvtHost::FixNotRespondingHost] Returning false since host is already fixed!

2014-03-14T22:14:33.127Z [02876 warning 'vpxdvpxdInvtHostCnx'] [VpxdInvtHostSyncHostLRO] Failed to fix not responding host host-2401

2014-03-14T22:14:33.127Z [02876 warning 'vpxdvpxdInvtHostCnx'] [VpxdInvtHostSyncHostLRO] Connection not alive for host host-2401

2014-03-14T22:14:33.127Z [02876 error 'vpxdvpxdInvtHostCnx'] [VpxdInvtHostSyncHostLRO] FixNotRespondingHost failed for host host-2401, marking host as notResponding

2014-03-14T22:14:33.147Z [02876 warning 'vpxdvpxdMoHost'] [HostMo] host connection state changed to [NO_RESPONSE] for host-2401

I see such errors multiple times for multiple hosts. I suggest you check the following in the decending priority order:

1. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=205405...

2. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100575...

3. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=204063...

But primarily I would suggest you to contact VMware Support at this stage. If the first link is correct (you need to check your vCenter Version and patch) then before installing any patch you should check with the support team.

Another point, since the problem is with only one host (but in the logs I could see that problem to be with multiple hosts, so you should get the same error for other hosts as well).

[pvevers]

Many thanks for your assistance here. Will proceed with this accordingly and have also raised with VMWare Support.

Regards