Dropped packets showing in ESXTOP

[HendersonD]

We have about 30 Windows VMs under vSphere 5.1. I am using esxtop to monitor the network and I see some odd behavior. Under the %DroppedPacketsReceived (%DRPRX) column you can see the issue. This column will show zeros for all VMs for a few seconds then all the VMs will show packet loss for a few seconds, then back to zero. I opened a ticket with VMWare and a tech worked on it for about 2 hours and could not find the root cause. He asked me to do some packet captures which I have tried through port-mirroring on a real port as well as port-mirroring on my DVSwitch. So far I cannot even find who is sending these packets. Since all VMs show dropped packets at the same time I am thinking some type of broadcast or multicast transmission.

Any ideas on how I can solve this problem?

[tomtom901]

Perhaps installing Wireshark on one of the VM's can help? Do all the VM's show this? Even a newly created?

[HendersonD]

If you look at the screen shot above, one of my VMs is a Win7 machine with Wireshark installed. It shows 40% packet loss in the last column. When I run the capture I do not see anything obvious but of course I am a bit new to using packet capture software so I am not exactly sure what I should be looking for when it comes to dropped packets. All VMs show this behavior

[admin]

Hello,

Are all the VM's Linux .?? or you have windows as well.?

Thanks,

Avinash

[tomtom901]

Other question, do the physical switch report any errors on the ports?

[HendersonD]

All the VMs are either Windows Server 2008 R2 (server cluster) or Win7 (View cluster). All the VMs show this same behavior with dropped packets

I checked a few physical switch ports looking for errors and did not see any

[HendersonD]

I figured it out

This thread about half way through talks about VDS Health Check sending out broadcast packets

https://communities.vmware.com/message/2280450

I disabled the Health Check and the dropped packets vanished. Any idea why a VM would drop packets from the VDS health check?

[tomtom901]

Are your VM's in the same broadcast domain as the host's management VMkernel?

[HendersonD]

They are not. I have a separate vswitch that handles just management traffic across vlan 123. This vswitch is attached to two 1GB Cisco switches. I have two other 10GB physical switches that handle VM traffic (vlan 125), iSCSI traffic (vlan 126), and vMotion (vlan 127). I have a dvSwitch setup for this purpose. I am wondering if my dvSwitch is setup properly.

• When I had the dvSwitch Health Check enabled, the reporting never worked quite right. I got the green checkmarks (setup correct) for some parameters and just a ? for others
• When the health check is on I see dropped packets on every VM. I do not see dropped packets in ESXTOP on vmnics or vkernel ports, just on VMs