LeslieBNS9
Enthusiast
Enthusiast

We are also seeing a lot of these errors on our All Flash vSAN environment. We've been doing some testing and think we have narrowed down the issue.

We have 6 hosts with the following configuration..

SuperMicro 1028U-TR4+

2xIntel E5-2680v4

512GB RAM

X710-DA2 10GB Network Adapters (Dedicated for vSAN, not shared)

Cisco 3548 Switches (Dedicated for vSAN, not shared)

We went through different drives/firmware on our X710, but so far none of that has made a difference.

We noticed on our Cisco switch that all of the interfaces connected to our vSAN were having discards on a regular basis (multiple times every hour). We opened a support case with Cisco to troubleshoot this and found that ALL of our vSAN ports have bursts of traffic that are filling up the output buffers on the switch. During these bursts/full buffers the switch discards the packets.

So I would check on your switches to see if you are having any packet discards.

At this point Cisco is recommending we move to a deep buffer switch. I spoke with VMWare support to see if there is a specific switch they recommend (or buffers), but they said they just require a 10Gb switch. I find this frustrating as we have 2 expensive switches we are only using 6 ports on and may not be able to add any more hosts to.

Ethernet1/2 queuing information:

    qos-group  sched-type  oper-bandwidth

        0       WRR            100

    Multicast statistics:

        Mcast pkts dropped                      : 0

    Unicast statistics:

    qos-group 0

    HW MTU: 16356 (16356 configured)

    drop-type: drop, xon: 0, xoff: 0

    Statistics:

        Ucast pkts dropped                      : 180616

Ethernet1/2 is up

Dedicated Interface

  Hardware: 100/1000/10000 Ethernet, address: 00d7.8faa.cf09 (bia 00d7.8faa.cf09)

  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

  reliability 255/255, txload 2/255, rxload 4/255

  Encapsulation ARPA

  Port mode is access

  full-duplex, 10 Gb/s, media type is 10G

  Beacon is turned off

  Input flow-control is off, output flow-control is off

  Rate mode is dedicated

  Switchport monitor is off

  EtherType is 0x8100

  Last link flapped 4d12h

  Last clearing of "show interface" counters 3d23h

  0 interface resets

  Load-Interval #1: 30 seconds

  30 seconds input rate 98177624 bits/sec, 4262 packets/sec

  30 seconds output rate 124356600 bits/sec, 4302 packets/sec

  Load-Interval #2: 5 minute (300 seconds)

    input rate 163.09 Mbps, 6.20 Kpps; output rate 113.03 Mbps, 6.33 Kpps

  RX

    2620601947 unicast packets  5716 multicast packets  335 broadcast packets

    2620612576 input packets  10625804438347 bytes

    1353181073 jumbo packets  0 storm suppression bytes

    0 runts  0 giants  0 CRC  0 no buffer

    0 input error  0 short frame  0 overrun   0 underrun  0 ignored

    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop

    0 input with dribble  0 input discard

    0 Rx pause

  TX

    2619585440 unicast packets  0 multicast packets  2452 broadcast packets

    2619587892 output packets  9072740199246 bytes

    1162617883 jumbo packets

    0 output errors  0 collision  0 deferred  0 late collision

    0 lost carrier  0 no carrier  0 babble 180616 output discard

    0 Tx pause

Reply
0 Kudos