ESXi 4.1 host disconnectivity issue

[Naresh_Dahagam]

We have vCenter Server 4.1 installed on a VM on Hyper-V,  vCenter server is added into a domain and this vCenter server is managing 28 ESXi 4.1 Enterprise Plus license hosts installed on Dell M610, ESXi hosts are not added in the domain.

Issue: we are facing intermittent ESXi 4.1 host disconnectivity issues (Disconnected or Not responding), when hosts disconnect some times it pings and sometimes not, sometimes VMs of that host are accessible and sometimes not and the host does not get connected from vSphere Client individually. Restarting the Management agents / reatsrting management network never resolved the issue, attached is the error message we get for all hosts whenver we try to reconnect the disconnected host. When this issue occurs the only thing we resort is to reboot the host, and when we reboot the host it works perfectly from vCenter or from vSphere Client.

We are facing lot of problems because of this issue, any help would be immensely appreciated.

Thank you.

Naresh Dahagam.

[Sreejesh_D]

welcome to the communities!!

how many hosts have this issue?

Looks like tis a host management network issue, since you said at the time of disconnect from VC the host is not responding for ping.

there will be different reason for management network failure.

can you check the events at /var/log/messages at the time of host disconnect? you can ilo to the host and check from ESXi troubleshooting console.

[vTagion]

Are the ESXi hosts nested virtual hosts or are they each physical? I agree it sounds like a host management network issue. I would also think to run wireshark before/during the issue at hand?

[Naresh_Dahagam]

Around 10 hosts are facing this issue.

We have 2 disconnected hosts now, and one host responds to ping and the other does not.

i have rebooted the host just a while ago and as i said it is working fine, pls find the attachments for the logs. But we can not reboot the host every couple of days.

i followed every relevant KB but no luck so far.

Thank you.

[Naresh_Dahagam]

No, they are not nested virtual hosts.

All the hosts are Dell Balde servers.

Will be pleased to have any suggestions to resolve this issue.

Thanks.

[a_nut_in]

Question:

1. When you say the hosts are not pinging - are they not pingable from the VC machine or any other physical machines as well?
2. Do you have any physical machines (windows/linux/does not matter) connected to the same physical switch on the same network segment and vlan as the ESX hosts and have you tried pinging from there? Does it respond?
3. On the ESXi console - during the time of the disconnect - what does ALT + F12 show on screen? Can a screenshot be shared?
4. On the ESX host that is having the issue try the following commands and share their output

#gunzip /var/run/log/*.gz

#cat /var/run/log/vmkernel* | grep -i mptsas | less

#cat /var/run/log/vmkernel* | grep -i esr | less

#cat /var/run/log/vmkwarning* | grep -i mptsas | less

#cat /var/run/log/vmkwarning* | grep -i esr

Also, what is the hardware of the hosts?

What is the current BIOS level?

Regards

a