VMware Cloud Community
impranayk
Enthusiast
Enthusiast

Unable to add host in Virtual Center

We are unable to add one ESXi host in VC. Giving attached error.

Error - ESXi Host Add.jpg

VPXD is showing below logs.

icate verification is disabled, so connection will proceed despite the error

2014-11-08T06:23:40.971+10:00 [10296 verbose 'SSL SoapAdapter.HTTPService'] User agent is 'VMware vim-java 1.0'

2014-11-08T06:23:40.971+10:00 [10296 verbose 'Default'] [VpxVmomi] Invoking [waitForUpdatesEx] on [vmodl.query.PropertyCollector:propertyCollector] session [528e6787-82ab-bf4e-3ccb-4e8d652b50e7(521c762d-2875-a801-43a0-0f2ed2e69621)]

2014-11-08T06:23:40.987+10:00 [03520 verbose 'Default' opID=8FD16342-00000205] [VpxVim] Local and Remote Version are the same. Talking with version vim.version.version7

2014-11-08T06:23:41.033+10:00 [13332 verbose 'DAS' opID=SWI-80e21c46] [FdmManager::SendPendingFdmUpdates] No master to send updates to. Will retry.

2014-11-08T06:23:41.111+10:00 [13652 warning 'ProxySvc Req54900'] Connection to localhost:31000 failed with error class Vmacore::SystemException(No connection could be made because the target machine actively refused it. ).

2014-11-08T06:23:41.127+10:00 [13652 verbose 'ProxySvc Req54903'] New proxy client TCPStreamWin32(socket=TCP(fd=8068) local=127.0.0.1:80,  peer=127.0.0.1:60379)

2014-11-08T06:23:43.077+10:00 [10296 verbose 'SoapAdapter.HTTPService'] User agent is 'VMware VI Client/4.0.0'

2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxVmomi] Invoking [retrieveContents] on [vmodl.query.PropertyCollector:propertyCollector] session [522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f(5243ef82-3fe2-dd25-5e17-ad56685d2cbd)]

2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxLroList::ThreadMainEntry] Unlocked _propProvider to call into external code

2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxLRO::LroMain] Invoking _currentMain

2014-11-08T06:23:43.077+10:00 [13652 info 'Default' opID=D5520D1B-000010C8] [VpxLRO] -- BEGIN task-internal-79722 --  -- vmodl.query.PropertyCollector.retrieveContents -- 522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f(5243ef82-3fe2-dd25-5e17-ad56685d2cbd)

2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default' opID=D5520D1B-000010C8] [VpxVmomi] Invoke done: vmodl.query.PropertyCollector.retrieveContents session: 522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f 

2014-11-08T06:23:43.077+10:00 [13652 verbose 'SoapAdapter.HTTPService' opID=D5520D1B-000010C8] HTTP Response: Complete (processed 1131 bytes)

2014-11-08T06:23:43.077+10:00 [13652 info 'Default' opID=D5520D1B-000010C8] [VpxLRO] -- FINISH task-internal-79722 --  -- vmodl.query.PropertyCollector.retrieveContents -- 522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f(5243ef82-3fe2-dd25-5e17-ad56685d2cbd)

2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxLRO::ClearCurrentMain] Clearing _currentMain

2014-11-08T06:23:44.325+10:00 [13872 verbose 'SoapAdapter.HTTPService' opID=c1bade25] HTTP Response: Complete (processed 1004724 bytes)

2014-11-08T06:23:44.653+10:00 [13872 info 'Default' opID=c1bade25] [VpxLRO] -- FINISH task-internal-79718 --  -- vmodl.query.PropertyCollector.retrieveContents -- 525144ec-a5fb-44c0-d24a-1a182c7c7afb(5273c6e1-09a6-3392-72a2-15b32eb62009)

2014-11-08T06:23:44.653+10:00 [13872 verbose 'Default'] [VpxLRO::ClearCurrentMain] Clearing _currentMain

-------------------------------------------------------------------------
Follow me @ www.vmwareinsight.com
Please consider marking this answer "correct" or "helpful" if you found it useful

Pranay Jha | Blog: http://vmwareinsight.com
vExpert 2016/2017, VCAP5-DCD/DCA, VCP5-DCV, VCA-Cloud, VCE-CIA, MCSE, MCITP
Tags (4)
Reply
0 Kudos
15 Replies
bspagna89
Hot Shot
Hot Shot

From vCenter, can that host be resolved/pinged? Not the solution, but have you tried adding it via IP and seeing if that works?

New blog - https://virtualizeme.org/
Reply
0 Kudos
impranayk
Enthusiast
Enthusiast

Yes, We can ping it from both side, VC to host and vice versa.

Also tried adding it by IP as well as FQDN. No Luck!

-------------------------------------------------------------------------
Follow me @ www.vmwareinsight.com
Please consider marking this answer "correct" or "helpful" if you found it useful

Pranay Jha | Blog: http://vmwareinsight.com
vExpert 2016/2017, VCAP5-DCD/DCA, VCP5-DCV, VCA-Cloud, VCE-CIA, MCSE, MCITP
Reply
0 Kudos
bspagna89
Hot Shot
Hot Shot

You can try restarting the management agents. See this for more info. If you have console access, its probably easiest to login and perform it from there.

VMware KB: Restarting the Management agents on an ESXi or ESX host

New blog - https://virtualizeme.org/
Reply
0 Kudos
admin
Immortal
Immortal

Are you able to directly connect to the ESXi host using VI client.

Can you paste the hostd logs ?

Reply
0 Kudos
Sateesh_vCloud

It will little crazy to hear but ESXi OS Re-installation will be fastest solution

------------------------------------------------------------------------- Follow me @ www.vmwareguruz.com Please consider marking this answer "correct" or "helpful" if you found it useful T. Sateesh VCIX-NV, VCAP 5-DCA/DCD,VCP 6-NV,VCP 5 DCV/Cloud/DT, ZCP IBM India Pvt. Ltd
Reply
0 Kudos
Burak201110141
Enthusiast
Enthusiast

Can you chek host bios date and time...
If Bios date and time wrong, sometimes i see that issue...

------------------------------------------------------------------ http://burakvardar.wordpress.com
Reply
0 Kudos
Dee006
Hot Shot
Hot Shot

Hi,

2014-11-08T06:23:41.111+10:00 [13652 warning 'ProxySvc Req54900'] Connection to localhost:31000 failed with error class Vmacore::SystemException(No connection could be made because the target machine actively refused it. ).

Can you verify the Port number are open in the Vcenter machine and try to restart the Vcenter services in vcenter

Reply
0 Kudos
impranayk
Enthusiast
Enthusiast

  • BIOS Date & Time is correct.
  • We couldn't connect this particular ESXi host from VI client as well.
  • We have checked ports and it's already opened.

When I logged into DCUI, "Lockdown Mode" option is grayed out.

Anything else what we can try?

-------------------------------------------------------------------------
Follow me @ www.vmwareinsight.com
Please consider marking this answer "correct" or "helpful" if you found it useful

Pranay Jha | Blog: http://vmwareinsight.com
vExpert 2016/2017, VCAP5-DCD/DCA, VCP5-DCV, VCA-Cloud, VCE-CIA, MCSE, MCITP
Reply
0 Kudos
impranayk
Enthusiast
Enthusiast

"/var/log/hostd.log" shows below logs while adding to VC.

2014-11-09T22:32:57.043Z [2D681B90 verbose 'Proxysvc Req00005'] New proxy client SSL(TCP(local=10.228.10.172:443, peer=10.228.2.93:59047))

2014-11-09T22:33:00.779Z [FFED1AC0 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root

2014-11-09T22:33:32.474Z [2D681B90 verbose 'ResourcePool ha-root-pool'] Root pool capacity changed from 32196MHz/254530MB to 32196MHz/254529MB

2014-11-09T22:33:36.587Z [2D681B90 verbose 'DvsManager'] PersistAllDvsInfo called

2014-11-09T22:33:46.008Z [2D681B90 verbose 'Default'] Timed out reading between HTTP requests. : Read timeout after approximately 50000ms. Closing stream TCP(local=127.0.0.1:8307, peer=127.0.0.1:63341)

2014-11-09T22:34:32.476Z [2DAC2B90 verbose 'ResourcePool ha-root-pool'] Root pool capacity changed from 32196MHz/254529MB to 32196MHz/254530MB

-------------------------------------------------------------------------
Follow me @ www.vmwareinsight.com
Please consider marking this answer "correct" or "helpful" if you found it useful

Pranay Jha | Blog: http://vmwareinsight.com
vExpert 2016/2017, VCAP5-DCD/DCA, VCP5-DCV, VCA-Cloud, VCE-CIA, MCSE, MCITP
Reply
0 Kudos
impranayk
Enthusiast
Enthusiast

Also, Just to add on this,

  • I cannot ping Default Gateway from ESXi host, however same gateway is reachable from other host in same configuration.
  • I can ping this ESXi host from VC and other machines.
  • I have cross checked IP configuration and all, no issue found.

Regards,

Pranay

-------------------------------------------------------------------------
Follow me @ www.vmwareinsight.com
Please consider marking this answer "correct" or "helpful" if you found it useful

Pranay Jha | Blog: http://vmwareinsight.com
vExpert 2016/2017, VCAP5-DCD/DCA, VCP5-DCV, VCA-Cloud, VCE-CIA, MCSE, MCITP
Reply
0 Kudos
admin
Immortal
Immortal

Seems network issue between vc and the host .

Are  you able to telnet 902 and 443 port  from vc.

Reply
0 Kudos
impranayk
Enthusiast
Enthusiast

Both port 443 & 902 are opened.

Default Gateway is not reachable from host, however same gateway is reachable from other hosts in cluster.

Also, I can ping few IPs in same segment where few are not pingable. Though the same IP addresses are reachable from other ESXi hosts.

-------------------------------------------------------------------------
Follow me @ www.vmwareinsight.com
Please consider marking this answer "correct" or "helpful" if you found it useful

Pranay Jha | Blog: http://vmwareinsight.com
vExpert 2016/2017, VCAP5-DCD/DCA, VCP5-DCV, VCA-Cloud, VCE-CIA, MCSE, MCITP
Reply
0 Kudos
Burak201110141
Enthusiast
Enthusiast

Hello,
How many uplink connected on the management vswitch?

If its 2, can you try only 1 uplink...

------------------------------------------------------------------ http://burakvardar.wordpress.com
Reply
0 Kudos
bspagna89
Hot Shot
Hot Shot

Hi,

Would it be possible to check the networking configuration? If you're using cisco switches with an etherchannel config, try disabling the etherchannel for now until getting the host into vCenter.

For now, I'd try using one NIC enabled in VMware with the proper VLAN for management and see if you can ping the gateway. You can also compare the configuration between a working host and this problematic host to see if everything matches.

Hope this helps.

New blog - https://virtualizeme.org/
Reply
0 Kudos
Sateesh_vCloud

From all the comments I understand that you are confident that ESXi host has Networking problem with below symptoms:

Assumptions:

You have identical network configuration for this ESX box like other working servers

You have PNIC's enabled with required VLAN's same as other working servers

vSwitch 0/ Management vSwitch - has valid PNICs enabled

Next steps:

Confirm Network Adapters are plugged into proper Physical Network switch port

Valid drivers are installed for Network Adapters - Try to update the new drivers or firmware as per compatability matrix

Physical Network switch port has required settings for VMware Virtualization like disable STP, configure Portfast


Replace the Network cables if they found faulty


Run full diagnostics for the server HW and open a case with Hardware vendor

------------------------------------------------------------------------- Follow me @ www.vmwareguruz.com Please consider marking this answer "correct" or "helpful" if you found it useful T. Sateesh VCIX-NV, VCAP 5-DCA/DCD,VCP 6-NV,VCP 5 DCV/Cloud/DT, ZCP IBM India Pvt. Ltd
Reply
0 Kudos