Guest Dropped Packets (Received) ?

[eversys]

We have noticed that our new monitoring server (n-central) which checks all of our vmware esxi 4.x & 5.x servers, complains that the threshold for "Guest Dropped Packets (Received)" on our virtual machines has failed. Our vmware servers seem to be running OK, just a little concerned what this service status shows.

Any ideas? Thanks, Oliver

[impranayk]

Can you check Network Load Balancing in vSwitch.

If it's configured for IP hash, try changing it to Virtual Port ID. It should fix packet drop issue.

[jackchentoronto]

checked the load balancing configuration and it's configured with "Route based on the originating virtual port ID" ( I guess this is the default setting? ), two adapters are configured as active and standby for this vswitch.

For a not very active VM port:

esxcli network port stats get -p 33554441

Packet statistics for port 33554441

   Packets received: 512363

   Packets sent: 11019

   Bytes received: 44523546

   Bytes sent: 856446

   Broadcast packets received: 499440

   Broadcast packets sent: 10

   Multicast packets received: 3112

   Multicast packets sent: 0

   Unicast packets received: 9811

   Unicast packets sent: 11009

   Receive packets dropped: 486956

   Transmit packets dropped: 0

It shows most broadcast packets are dropped, but in the VM level, when I run tcpdump, it actually shows those broadcast packets, so maybe this is a bug in ESXi ?

[jackchentoronto]

I changed a VM's NIC from e1000 to VMXNET3; changed its ethernet0.emuRxMode to 1, it didn't make any difference.

When I booted up this VM, esxcli shows:

esxcli network port stats get -p 33554460 | grep -i rece

   Packets received: 1755

   Bytes received: 147212

   Broadcast packets received: 1688

   Multicast packets received: 9

   Unicast packets received: 58

   Receive packets dropped: 1642

Inside this VM, ethtool shows:

ethtool -S eno16780032 | grep  -i "pkts rx"

ucast pkts rx: 58

mcast pkts rx: 9

bcast pkts rx: 1690

and there is no dropped packets ( ethtool run seconds later than the esxcli command ) .

So the VM actually received all the broadcast/multicast packets, but somehow ESXi shows they were dropped.

[karthikelango]

Hi,

What version of NIC you are using in your Guest?

Have you tried upgrading the version to vmxnet?

[chr1s86]

Same expirience here.

Cisco UCS Blades, Nexus 1000V, ESX 5.5 Build 2403361

VMs are mixed. Windows and Linux with E1000 and VMXNET3.

[jackchentoronto]

For anybody who sees packet drop on esxcli but not esxtop, it's not real packet drop, but a ESXi bug:

VMware KB: vCenter Server 5.1/5.5/6.0 performance charts report dropped network packets

Seems it's fixed on 5.1 and 5.5 now, but still no fix on 6.0.