ESXi 4.0.0 Build 261974 - Network Connectivity issues

[enableit]

Hello all,

I am having an issue with 3 Guests (all Windows Server 2008 R2, only 2 have VM Ware tools installed) constantly losing network connectivity.

I have been searching for an answer to this issue for days now and have tried some steps provided in a few other discussions but still having the same issue.

Attached is a screen shot of my network config on the host.

The host is connected to a Sonic Wall 1260, port security is turned off and both ports that the host is connected to are set to Auto Negotiate and are showing as 100mbps Full.

I have recently patched and upgraded the Host using the Host Update utility and the esxupdate utilities.

And now im lost....Any help would be greatly appreciated at this time.

Cheers!

[RParker]

Sounds like an autoconfiguration problem of the physical NIC and Sonic Wall. if it's just the VM and other VM's on the same vSwitch on ESX are not losing connection, then try change the NIC type, like VMXNET2 or VMXNET3

[oz]

It might be worth turning off Autonegotiation and nailing both the pNIC and the Sonicwall port to the speed/duplex that you can support. If all 3 VMs lose connectivity that seems to incidate a common source... like the pNIC. Does the Sonicwall log show a drop/reconnect event?

Good luck.

[enableit]

Nope all VM's are losing the connection at the exact same time. Ive been running a ping against all the guests and host and the Host never goes down, all the VM's conncetions drop at the same time....first it says the Request timed out then goes on to Host is unreachable. Does this for about a minute then comes back. Everything is fine for 3-5 minutes then drops again.

oz: Tried setting the nics and port to Full 100 and still the same issue occuring.

[oz]

You said, "Ive been running a ping against all the guests and host and the Host

never goes down, all the VM's conncetions drop at the same time."

Q1) Were you pinging the host on the .40 or the .42 address, or both?

Q2) Since the 3 VMs are spread over 2 pNICs, I'd be looking for common events upstream from the pNICs... presumably the Sonicwall. Anything useful in its log?

Q3) based on your answer to the first question, is there any chance that you have a duplicate IP somewhere, causing the sonicwall to see a routing flap between the rogue/duplicate and your physical NIC(s)?

[DSTAVERT]

We don't know what the hardware platform is or what NIC is in the ESXi host. Simple things to try. Replace cables. Different switch port. I would put a switch in front of the sonicwall and connect the LAN and ESXi host to that switch. Make it a 1GB switch.

[DSTAVERT]

Have a look through log ESXi files. Download from File ->Export menu.

[enableit]

oz

Q1 - Running a ping on both the Host IP's

Q2 - Nothing unusual or unexpected

Q3 - Nope i have done a network scan and found that the IP addresses assigned to the host and the guests are only being used by those devices

What about the NIC's on the guest machines, do you have to configure them to match the 100 full duplex to match the host and firewall?

[enableit]

Hardware platform is a Dell Power Edge 2950, 1 Quad core CPU Xeon L5335, 12GB RAM, NICS are Broadcom NetXtreme II BCM5708 1000Base-T.

Downloading the Log Files now from the Host, anything in particular i should be looking for. I have this feeling ill be cross eyed 5 minutes after opening them

[Silverchenau]

Normally, you should check out vmkwarning, vmkernel, and hostd logs.

Can you try to plug in your nic to a normal switch? Assuming that physical switch is has Gigabyte port, will your nic get 1000 instead of 100?

Plug in your nic to another switch (not for SC nic), will they still drop off?

I don't think there is problem in ESX, more likely it's problem of your sonic firewall.

My Vmware blog: http://geeksilverblog.com

[enableit]

Ok I put a switch between the Firewall and the Host, started up one of the Guests and connection didnt drop. I started up another Guest and after about a minute the connection on both Guests dropped. Pinging against both the host IP's .40 and .42 they never drop.

Interestingly enough now if i do a nslookup against the hosts hostname it wont resolve, nslookup on the guests works no problem though....

[oz]

It would be interesting to have a single VM up for a longer period of time, to see whether drops tend to occur only when multi VMs running, or if they just occur less frequently with one VM.

Also, when you're in the condition where the host's name won't resolve, please describe where you're running the nslookup or dig command from. E.g. does nslookup on the host's command line behave differently than an nslookup run from workstation elsewhere on the network, for example.

Does the DNS server that is being queried live in the same network as the ESX hosts and the VMs, or does the DNS traffic have to traverse the firewall?

[enableit]

Interestingly enough...I have all 3 vm's running now and running a ping against them. #1 and #3 are losing their connection, but #2 isnt. The only differnce between them is VM Ware tools both #1 and #3 have them installed while #2 does not.

Im uninstalling from #3 right now and will keep monitoring.

[enableit]

For the nslookup i attempted to resolve the hstname from a workstation and directly from the Domain controller, same result each time:

cal-hst-01

Server: csi-cal-dc01.centuryservicesinc.com

Address: 192.168.8.5

DNS request timed out.

timeout was 2 seconds.

• Request to csi-cal-dc01.centuryservicesinc.com timed-out

Within the Host DNS Managment tool if i test the management network it can ping the DNS Servers but cannot resolve the hostname.

The Domain controllers are connected directly to the firewall, where as the Host goes through the switch i recently added.

[oz]

Did you mention what version of vmtools is/was installed?

Also, are your VMs hardware version 4 or 7?

[enableit]

Looking in the about section for VM Ware tools and it says 4.0.0 261974 - same as ESXi version

VM Ware hardware version 7

[enableit]

Ok so now I have all 3 VM's running.

#1 has vmware tools - keeps dropping network

#2 no vmware tools - keeps connection

#3 no vmware tools - keeps dropping connection

[Silverchenau]

hi

I believe you misunderstand my point.First of all, we need to find out what you mean of dropping.

After you connect your vm to new physical switch, when your VM drops off, can you see your vmnic disconnected as well from vCenter?

If it is not disconnected, it means your ESX is fine with physical connection to your physical swtich.

Problem maybe sits on your firewall. But that's network issue not ESX issue.

- Silver

My Vmware blog:

[Raamz]

you should check few things

1. recommended virtual adapter for this OS is

VMXNET 3

Supported

Third generation VMware virtual NIC

then check for this error if this applies to you.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101297...

install vmware tools on all guest and restart the OS and then check again for packet drops.

2.

few basic questions:

1.what changes you have done after which you are facing this problem.

2.What is the exact version of the guest operating system.

3.are you facing the ping drop issue with only this OS guest or all Guest OS.

also try to run tcpdump

tcpdump -i vswif0

and check packets is it no reply or something else

this will help you identify the issue better.

Happy to help you

cheers

Raamz

[enableit]

Ok i see now....No the vmnic in vsphere never shows as being down or disconnected. I will give Sonic Wall a call and see if they have any suggestions.

Thanks!