N7Vmacore15SystemExceptionE error: ESXi host live but disconnected in vCenter

[WillL]

Hi,

ESXi hosts are accessible via SSH and VMs are also working. But ESXi hosts are disconnected in vCenter.

@ hostd.log:

[2011-02-07 16:56:03.950 FFFD1E90 info 'Vimsvc'] [Auth]: User root                                                                                              

[2011-02-07 16:56:03.950 FFFD1E90 info 'ha-eventmgr'] Event 3 : User root@127.0.0.1 logged in                                                                   

[2011-02-07 16:56:05.457 3EFD5B90 verbose 'Proxysvc Req00006'] New proxy client TCP(local=127.0.0.1:60989, peer=127.0.0.1:80)                                                          

[2011-02-07 16:56:05.674 3EE40B90 verbose 'Ticket 52 e3 1b 60 89 8e 5a 96-b4 29 d7 9f 2f bb 8a c7'] Ticket invalidated                                                                                   

[2011-02-07 16:56:05.951 3EE40B90 verbose 'Proxysvc Req00007'] New proxy client SSL(TCP(local=127.0.0.1:57144, peer=127.0.0.1:0))                                                                    

[2011-02-07 16:56:05.951 3ECF7B90 warning 'Proxysvc Req00007'] Error reading from client while waiting for header: N7Vmacore15SystemExceptionE(Connection reset by peer)         

[2011-02-07 16:56:06.151 3ECF7B90 verbose 'Proxysvc Req00008'] New proxy client SSL(TCP(local=127.0.0.1:52388, peer=127.0.0.1:443))                

.............

............

[2011-02-07 17:11:09.718 3EE40B90 error 'App'] Failed to read header on stream TCP(local=127.0.0.1:59909, peer=127.0.0.1:0): N7Vmacore15SystemExceptionE(Connection reset by peer)                     

[2011-02-07 17:11:13.788 3F540B90 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root                                                                                                      

[2011-02-07 17:11:20.014 3EE81B90 verbose 'Statssvc'] HostCtl exception Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.   

......

[2011-02-07 17:16:10.339 3ECF7B90 error 'App'] Failed to read header on stream TCP(local=127.0.0.1:57997, peer=127.0.0.1:0): N7Vmacore15SystemExceptionE(Connection reset by peer)

[2011-02-07 17:16:20.014 3EE81B90 verbose 'Statssvc'] HostCtl exception Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.

......

[2011-02-07 17:21:10.949 FFFD1E90 error 'App'] Failed to read header on stream TCP(local=127.0.0.1:63284, peer=127.0.0.1:0): N7Vmacore15SystemExceptionE(Connection reset by peer)

[2011-02-07 17:21:20.014 3ECF7B90 verbose 'Statssvc'] HostCtl exception Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.            

@ vpxa.log:

[2011-02-07 16:56:11.187 FFD2BB10 info 'SoapAdapter.HTTPService'] Http Service started: TCPServerSocket(ASYNC_ACCEPT, ipv4=TCP(fd=97 name=127.0.0.1:8090), ipv6=TCP(null))

[2011-02-07 16:56:11.187 FFD2BB10 info 'App'] [VpxVmomi] SOAP adapter started on port 8090                    

[2011-02-07 16:56:11.187 FFD2BB10 error 'App'] [VpxVmomi] Failed to create WebService socket: N7Vmacore15SystemExceptionE(File exists)

[2011-02-07 16:56:11.187 FFD2BB10 error 'App'] [VpxaVmomi] Failed to start VMOMI layer: File exists                                                                              

[00] rip 095cf383                                              

[01] rip 094b961e                                                                                                   

[02] rip 09528042                                                                                                                        

[03] rip 09484ee0                                                                                                          

[04] rip 094794e2                                                                                   

[05] rip 094738dd                                                        

[06] rip 09473c1e                                                                             

[07] rip 084dc4ba                                                                                                        

[08] rip 0851ead4                                                                          

[09] rip 0851f520                                                                                            

[10] rip 082b9105                                                                       

[11] rip 08255f6d                                                                             

[12] rip 082573f3                                                                                                                                       

[13] rip 08253791                                                                                                             

[14] rip 1b793f0c                                                                                             

[15] rip 08252c51                                                                                                

[2011-02-07 16:56:11.188 FFD2BB10 error 'App'] Failed to start modules

@ End of logs

Restarted management network and agent on one of ESXi, didn't resolve the issue.

Any suggestions?

Thanks.

William

[admin]

Hi

Have a look on this KB http://kb.vmware.com/kb/1014270 it may help you out.

Regards

[WillL]

DNS works fine.

All VMs are accessible, so do ESXi hosts, just disconected in vCenter and the error in logs.

Maybe reboot?

[WillL]

Rebooted still the same issue, reboot did take a long time.

Double checked DNS, everything works fine including forward/reversal resolution, FQDN and short hostname.

Really need some help!

[WillL]

Rebooted again, manually connect ESXi hosts to vCenter, everything is back online, not sure how and when this was fixed, I was expecting vCenter would auto connect to disconnected hosts. :smileycool: