-
15. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
JohnNicholsonVMW Mar 27, 2019 8:41 PM (in response to TolgaAsik)Nexus 55xx used a VoQ buffer system. Basically the (limited) buffer isn't split across all the ports in a shared pool. There's also very few queues in general per port.
It's not a great switch for high performance operations, and I believe CIsco abandoned that VoQ system in the 56xx.
If you want a deep buffer Cisco switch, get a C36180YC-R 8GB of buffer allocation for the ports.
The 55xx is a puny 470 for the default traffic class (it's 640KB but stuff by default is reserved for FCoE and other stuff).
The 3548 is interesting. IT can achieve some crazy low port to port lat, but that's really only a big deal with RDMA traffic. Might help in the future but not now.
-
16. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
wreedMH Mar 28, 2019 12:49 PM (in response to LeslieBNS9)We recently had a similar issue on a 4-node vSAN cluster and it turned out to be the x710 NICs. Switched all 4 nodes to Broadcom and latency is gone.
-
17. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
TolgaAsik Apr 5, 2019 12:38 PM (in response to JohnNicholsonVMW)Hi John,
Thanks for your switch recommendation. We will care them in next investment.
By the way second recommendation of Cisco related to my case is to increase connected ports to backbone switches. Because my servers connected to Nexus, Nexus connected to Catalyst6500 with 2 ports in port-channel. these 2 ports are congested ports, and so it is applying back pressure to end devices.
We will increase the count of ports or replace the switches.
-
18. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
wreedMH Apr 5, 2019 6:31 PM (in response to LeslieBNS9)Tolga,
What model NICs in your servers?
-
19. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
TolgaAsik Apr 29, 2019 11:01 AM (in response to wreedMH)Hi,
My NIC model installed on servers is
Model : Synergy 3820C 10/20Gb CNA
Firmware : 7.13.109.0 BC: 7.15.24
Driver : qfle3 - 1.0.77.2
8 nodes are running in streched VSAN cluster.
Nowadays I am planing to update them to 6.7 U2 and the latest HPE Synergy SPP bundle.
-
20. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
jo.strasser Apr 29, 2019 11:57 PM (in response to TolgaAsik)Hi Tolga,
Be aware with the latest SPP-2019.03 for HPE Synergy.
It is not completely certified for vSAN right now, only NVMe configurations.
If you are using SAS disks... don´t upgrade yet.
The vSAN certification for the new Smart Array Firmware 1.98 and the new driver is not completed right now.
BR/JO
-
21. Re: All-Flash vSAN Latency & Network Discards (Switching Recommendations)
TolgaAsik Apr 30, 2019 2:15 AM (in response to jo.strasser)Hi Johannes,
Thanks for your update.
I am in contact with VMware now.
My HPE Smart Array Card is not certified on Vmware VSAN Compatibility Guide for new firmware.
I am following up, after it is approval by VMware, I will apply immediately the latest HPE Synergy bundle to my all Synergy environment.
Model : P204i-c
Approved firmware is 1.65 for VSAN.