VMware Cloud Community
1995huskers
Contributor
Contributor

ESXTOP Excessive Packets Received

I was looking into some storage performance issues for our VMware  environment and I'm seeing some odd output from ESXTOP.  We  are running ESXi 5.0 (latest build) on Cisco UCS B200 M2 blades.  We connect to our  NetApp 3240 using NFS.

Below  is an example of what we are seeing in ESXTOP.  You'll see vmnic4 is  receiving a ton of traffic packets, while the kernel port (vmk3) that is  bound to vmnic4 is receiving a lot less.  There is nothing else using  vmnic4 on the host.  vmnic5 is also on the same NFS vlan as vmnic4 and  it is not receiving the traffic, so I don't think it's a broadcast storm  or anything.

Has anyone else seen anything like this before?  Is this normal?

We have about 15 hosts that are connected to this storage and I'm seeing the same behavior on all of them.

I  would assume the PKTRX/s received should be about the same on the port  (vmnic) as the vmkernel port (vmk3), assuming that port is used for nothing but the  kernel port.  Also, it is using its own VLAN for our NFS storage  traffic.

9-14-2012 4-31-27 PM.jpg

Reply
0 Kudos
1 Reply
titaniumlegs
Enthusiast
Enthusiast

The MbRX/s is pretty close, but PKTRX is 14x on the vmnic.  In my lab I see more on the vmnic than the vmk, but not by that much.

   PORT-ID              USED-BY  TEAM-PNIC DNAME              PKTTX/s  MbTX/s    PKTRX/s  MbRX/s %DRPTX %DRPRX
  16777217           Management        n/a vSwitch0              0.00    0.00       0.00    0.00   0.00   0.00
  16777218               vmnic0          - vSwitch0              6.20    0.02       3.34    0.00   0.00   0.00
  16777219                 vmk0     vmnic0 vSwitch0              3.34    0.01       2.38    0.00   0.00   0.00
  16777226                 vmk3     vmnic0 vSwitch0              0.00    0.00       0.48    0.00   0.00   0.00
  16777237 4199584:RHEL54x64vs5     vmnic0 vSwitch0              0.00    0.00       0.48    0.00   0.00   0.00
  16777238   4191400:esxiclone0     vmnic0 vSwitch0              0.00    0.00      10.01    0.02   0.00   0.00
  16777244        4216965:XPcm1     vmnic0 vSwitch0              0.00    0.00       0.48    0.00   0.00   0.00
  16777246        4203695:XPcm5     vmnic0 vSwitch0              0.00    0.00       0.48    0.00   0.00   0.00
  16777247   6933787:esxi51rtm1     vmnic0 vSwitch0              2.86    0.00       7.15    0.02   0.00   0.00
  33554433           Management        n/a vSwitch1              0.00    0.00       0.00    0.00   0.00   0.00
  33554434               vmnic1          - vSwitch1              0.00    0.00       1.43    0.00   0.00   0.00
  33554435                 vmk2     vmnic1 vSwitch1              0.00    0.00       0.00    0.00   0.00   0.00
  50331649           Management        n/a vSwitch2              0.00    0.00       0.00    0.00   0.00   0.00
  50331650               vmnic2          - vSwitch2              1.43    0.00    5989.07  199.66   0.00   0.00
  50331651               vmnic4          - vSwitch2           3036.98  344.76       3.81    0.00   0.00   0.00
  50331652                 vmk1     all(2) vSwitch2           3035.07  344.76    4531.86  198.95   0.00   0.00
  50331653 4199584:RHEL54x64vs5     all(2) vSwitch2              0.00    0.00       0.00    0.00   0.00   0.00
  50331654   4191400:esxiclone0     all(2) vSwitch2              3.81    0.00    4540.92    3.77   0.00  65.32
  50331655   4191400:esxiclone0     all(2) vSwitch2              0.00    0.00    4544.73    3.77   0.00  65.30
  50331656   6933787:esxi51rtm1     all(2) vSwitch2              0.00    0.00    4544.73    3.77   0.00  65.30
  50331657   6933787:esxi51rtm1     all(2) vSwitch2              0.00    0.00    4544.73    3.77   0.00  65.30

My first thought was something with Jumbo frames, but I think jumbo fragmentation and reassembly would be done in the pswitch.  Still, maybe check the jumbo settings on the dvSwitch and VMK.

Second, maybe ifstat on the filer might shed some light.  Use ifconfig -a to see which ethernet ports make up the VIF (now called ifgrp) then ifstat e2a or whatever to see the counters.  You can reset counters with ifstat <nic> -z or ifstat -a -z.

Share and enjoy! Peter If this helped you, please award points! Or beer. Or jump tickets.
Reply
0 Kudos