- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are also seeing a lot of these errors on our All Flash vSAN environment. We've been doing some testing and think we have narrowed down the issue.
We have 6 hosts with the following configuration..
SuperMicro 1028U-TR4+
2xIntel E5-2680v4
512GB RAM
X710-DA2 10GB Network Adapters (Dedicated for vSAN, not shared)
Cisco 3548 Switches (Dedicated for vSAN, not shared)
We went through different drives/firmware on our X710, but so far none of that has made a difference.
We noticed on our Cisco switch that all of the interfaces connected to our vSAN were having discards on a regular basis (multiple times every hour). We opened a support case with Cisco to troubleshoot this and found that ALL of our vSAN ports have bursts of traffic that are filling up the output buffers on the switch. During these bursts/full buffers the switch discards the packets.
So I would check on your switches to see if you are having any packet discards.
At this point Cisco is recommending we move to a deep buffer switch. I spoke with VMWare support to see if there is a specific switch they recommend (or buffers), but they said they just require a 10Gb switch. I find this frustrating as we have 2 expensive switches we are only using 6 ports on and may not be able to add any more hosts to.
Ethernet1/2 queuing information:
qos-group sched-type oper-bandwidth
0 WRR 100
Multicast statistics:
Mcast pkts dropped : 0
Unicast statistics:
qos-group 0
HW MTU: 16356 (16356 configured)
drop-type: drop, xon: 0, xoff: 0
Statistics:
Ucast pkts dropped : 180616
Ethernet1/2 is up
Dedicated Interface
Hardware: 100/1000/10000 Ethernet, address: 00d7.8faa.cf09 (bia 00d7.8faa.cf09)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 2/255, rxload 4/255
Encapsulation ARPA
Port mode is access
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 4d12h
Last clearing of "show interface" counters 3d23h
0 interface resets
Load-Interval #1: 30 seconds
30 seconds input rate 98177624 bits/sec, 4262 packets/sec
30 seconds output rate 124356600 bits/sec, 4302 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 163.09 Mbps, 6.20 Kpps; output rate 113.03 Mbps, 6.33 Kpps
RX
2620601947 unicast packets 5716 multicast packets 335 broadcast packets
2620612576 input packets 10625804438347 bytes
1353181073 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
2619585440 unicast packets 0 multicast packets 2452 broadcast packets
2619587892 output packets 9072740199246 bytes
1162617883 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 180616 output discard
0 Tx pause