VMware Cloud Community
Srinivasanm
Contributor
Contributor

Esx host disconnect Vcenter

Hi,

i have a HP server which is installed ESX4.0.0 build 164009.once i connect the host to vcenter couple of minutes later its disconnect from the vcenter.

even though i restarted the management service. still same issue. If i access the host directly using Vsphere client its working.

Any one faced same issue. please help me on the same.

Thanks & Regards,

Srinivasan M

Tags (1)
0 Kudos
11 Replies
scott28tt
VMware Employee
VMware Employee

Moderator note: moved to the correct forum area.


-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog
0 Kudos
dales123
Enthusiast
Enthusiast

What version of vcenter is (not that that makes too much difference). Also have you been moving around any storage recently (i.e. migrating from one storage platform to another, or just migrating luns and deleting old ones etc). How many other hosts are connected to vcenter if a few I take it they are not getting the same issue.

Is there anything interesting in the hosts events tab?

Regards

Dale

Kind Regards Dale VCP3+5
0 Kudos
Srinivasanm
Contributor
Contributor

Hi Dale,

   i am using vcenter 4.0 and in the same vcenter i am managing 60 hosts. Is there any limitation on this vcenter. in this there is no additional storage.i am using local storage.

Thanks & Regards,

Srinivasan M

0 Kudos
Srinivasanm
Contributor
Contributor

Hi Dale,

i observed on more thing is the whenever the host is disconnect from the vcenter that time i was getting alert "alert 'host connection and poer state' on host_name changed from GREEN to RED". some times later host is not reachable. Here all other host in the same network are working fine. only this host making trouble.below detailsfor your reference.

Alarm 'Host connection failure' on entity

glox-esx14 send SNMP trap
info
10/5/2012 11:56:39 AM
Alarm 'Host connection failure' on
glox-esx14 triggered an action
info
10/5/2012 11:56:39 AM
Alarm 'Host connection failure' on
glox-esx14 changed from Gray
to Gray
info
10/5/2012 11:56:39 AM
Alarm 'Host connection and power state' on glox-
esx14 changed from Green to
Red
info
10/5/2012 11:56:39 AM
Host is not responding
error
10/5/2012 11:56:38 AM

Thanks & Regards,

Srinivasan M

0 Kudos
dales123
Enthusiast
Enthusiast

Hi,

I think from memory per cluster there is a limit of 32 hosts in ESX4 (have a quick google for the esx4 config maximum guides which will give you definiative answers, however how going over the host limit manifests itself I'm not too sure. as you are possibly way over (if all 60 are in a single cluster) I would expect one or two more hosts to present the same issue if this were linked to the limit. What is the sql backend you are using for vcenter (is it express or a full blown off vcenter sql install).

Personally with 60 hosts the first thing I would try is basic cable swapouts or physical traffic monitoring are all the network blinky lights blinking as they should, or if not flattening the esx box and reinstalling esx on it from scratch to see if that helps, as your running local storage thoough you'll have to be careful if you have any non replicated vms etc on the box.

Kind Regards Dale VCP3+5
0 Kudos
irfans123
Enthusiast
Enthusiast

Hi Srinivas,

Please follow the below steps, you may fix the issue

  1. Disconnect and reconnect the host: 
    1. Right-click the ESX host in vCenter Server and click Disconnect.
    2. Reconnect the ESX host in vCenter Server.
  2. Restart the VirtualCenter agent on the ESX host: 
    1. Stop the vpxa service with the command:

      # service vmware-vpxa stop

      To stop the vpxa service in ESXi 5.0, run this command in the Tech Support mode.

      # /etc/init.d/vpxa stop
    2. Open the /etc/opt/vmware/vpxa/vpxa.cfg file in a text editor and correct the IP address of the ESX host.
    3. Start the vpxa service with the command:

      # service vmware-vpxa start

      To start the vpxa service in ESXi 5.0, run this command in the Tech Support mode.

      # /etc/init.d/vpxa start

      Note: For VMware ESXi you may need to restart all of the management agents. For more information, see Restarting the Management agents on an ESX or ESXi Server (1003490) .
  3. Rebuild the VirtualCenter agent configuration file on the ESX host:

    Caution:  This step removes all host and virtual machine historical statistics  counters. If the VMware ESX host is running virtual machines that  started up in an EVC enabled environment, you may not be able to add the  host back without shutting down the virtual machines first.
    1. Right-click the ESX host in vCenter Server and click Disconnect.
    2. Remove the disconnected ESX host from vCenter Server.

      Note: It has been  reported that in some instances it may be necessary to disable SSL  certificates in the vCenter Server > Administration > vCenter Server Settings > SSL Settings for the update to vpxd.cfg to work.The SSL  certificates can be re-enabled once the ESX host has been re-added back to the vCenter Server inventory.

    3. Backup the vpxa.cfg file with the command:

      Note: In ESXi 5.0, the vpxa.cfg configuration file is located at /etc/vmware/vpxa/.

      # mv /etc/vmware/vpxa/vpxa.cfg /etc/vmware/vpxa/vpxa.oldcfg
    4. Add the disconnected ESX host to the vCenter Server inventory.
    5. Run this command to examine the contents of /etc/opt/vmware/vpxa/vpxa.cfg and confirm that the host IP address is correct:

      # cat /etc/opt/vmware/vpxa/vpxa.cfg

      In ESXi 5.0, run this command:

      # cat /etc/vmware/vpxa/vpxd.cfg
0 Kudos
Srinivasanm
Contributor
Contributor

Hi,

After doing this activity, still same problem.

Thanks &Regards,

Srinivasan M

0 Kudos
irfans123
Enthusiast
Enthusiast

Hi Srinivasan,

Please verfiy or check for the root cause by following the below steps and let us know where you are struck to proceed further and please mention the errors if ur getting...

1.Verify that the ESX host is in a powered on state.
2.Verify if the ESX host can be reconnected or if reconnecting the ESX host resolves the issue.
3.Verify if the ESX host is able to respond back to vCenter Server at the correct IP address. If vCenter Server does not receive heartbeats from the ESX host, it goes into a not responding state. To verify if the correct Managed IP Address is set, see Verifying the vCenter Server Managed IP Address (1008030)
A. To check if a managed IP has been set on vCenter Server 4.x and above:
    From the vSphere Client, log in to vCenter Server
    Navigate to Administration > vCenter Server Settings > Runtime Settings and review the Managed IP Address setting.
    Verify that the address is correct. If not, correct the entry and click OK to save your changes and close the dialog.
    Remove the ESX/ESXi host from the vCenter Inventory.
    Add the ESX/ESXi host back to vCenter Inventory.
4.Verify that network connectivity exists from vCenter Server to the ESX host. For more information, see Testing network connectivity with the Ping command (1003486) .
5.Verify that you can connect from vCenter Server to the ESX host on TCP/UDP port 902. If the ESX was upgraded from version 2.x and you cannot connect on port 902, then verify if you can connect on port 905. For more information, see Testing port connectivity with Telnet (1003487) .

C:\>telnet server 25
    Connecting...
    220 server ESMTP Sendmail 8.13.3/8.13.3;
    Connecting to port 443 on the vCenter Server:
    C:\>telnet server 443
    Connecting...
6.Verify that the ESX management service vmware-hostd is running. For more information, see Verifying that the Management Service is running on an ESX host (1003494) and Troubleshooting vmware-hostd service if it fails or stops responding (1002849).
Run this command:
ps -ef | grep hostd | grep -v grep
The output appears similar to below if vmware-hostd is running:
[root@server]# ps -ef | grep hostd | grep -v grep
root     23204     1  0 15:27 ?        00:00:00 /bin/sh /usr/bin/vmware-watchdog -s hostd -u 60 -q 5 -c /usr/sbin/hostd-support /usr/sbin/vmware-hostd -u
root     23209 23204  1 15:27 ?        00:04:23 /usr/lib/vmware/hostd/vmware-hostd /etc/vmware/hostd/config.xml -u
[root@server]#
The output appears similar to the below if vmware-hostd is not running:
[root@server]# ps -ef | grep hostd | grep -v grep
[root@server]#
7. Verify that the VirtualCenter agent service vmware-vpxa is running. For more information, see Verifying that the vCenter Server Agent Service is running on an ESX host (1003495) .
Type ps -ef | grep vpxa ( if no output means service not running )
8.   Verify that the xinetd service is running. If xinetd is not running, authentication may fail. For more information, see Determining if the xinetd service is running on an ESX host (1007323) .
#service xinetd status            #service xinetd start
9. Verify if restarting the ESX Management Agents resolves the issue. For more information, see Restarting the Management agents on an ESX or ESXi Server (1003490) .
#service mgmt-vmware restart and #service vmware-vpxa restart
10. Verify that no processes are over utilizing the resources on the Service Console. For more information, see Checking for resource starvation of the ESX Service Console(1003496) .
Type top.  and To exit top, press Q
11.    ESX hosts can disconnect from vCenter Server due to underlying storage issues. To investigate further, see Identifying Fibre Channel, iSCSI, and NFS storage issues on ESX/ESXi hosts (1003659)

0 Kudos
UmeshAhuja
Commander
Commander

Hi,

Check with this --> VMware KB: Diagnosing an ESX Server that is Disconnected or Not Responding in VirtualCenter
and might fix it with this --> VMware KB: ESX host continually disconnects from the vCenter Server following an upgrade to ESX 4.0

Thanks n Regards
Umesh Ahuja

If your query resolved then please consider awarding points by correct or helpful marking.
0 Kudos
yasona78
Contributor
Contributor

Helloo,

we had the same issue , and we concluded that we have to open ( 902 TCP and 902 UDP)  in adition to 903 , 8080 and 443 .

0 Kudos
lvaibhavt
Hot Shot
Hot Shot

You mentioned that you were getting an alert as well

host connection and power state changed from green to red

Add this entry in the article and let us know if this helps

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=102021...

0 Kudos