VMware Cloud Community
jsisen
Contributor
Contributor

How to run max Bandwidth when Teaming NIC port 2x10G Cisco to Esxi vSphere 8

Dear Vmware Technical

I have 1 host esxi vsphere 8.0 and have 2x 10G vmnics. I configured the standard switch (vS10G) running Teaming 2x10G Mode Route base IP Hash and trunking vlan to cisco 9300 2xport 10G. I have also configured ethernetchannel port and port-channel load-balance src-dst-ip running mode on. When I apply Port group to the VM, it doesn't run at the maximum speed of 20G, but only runs somewhere at 5.6G/10G speed. I used the tool iperf test as shown below.

2023-08-10_17-30-01.png

2023-08-11_09-07-44.png

 My config in cisco

port-channel load-balance src-dst-ip

int po1
description PO1-ESX01
switchport mode trunk
int te1/1/5
description PO1-ESX01-Trunk-VMNIC6
switchport mode trunk
channel-group 1 mode on

int te2/1/5
description PO1-ESX01-Trunk-VMNIC7
switchport mode trunk
channel-group 1 mode on

How to run max speed using 2 ports 10G=20G.

Thanks

 

Labels (1)
Reply
0 Kudos
10 Replies
Brisk
Enthusiast
Enthusiast

To test the maximum bandwidth, you need to use multiple streams. Just because you're using 2 10 gig nics in a bond, doesn't mean you can then see throughputs of 20 gbps. You are still limited to 2x 10 gig streams.

You also need to take into account some overhead. You're running iperf in a windows host - is it using the vmxnet3 adapter?

I try to avoid LACP as much as possible towards my host and just let vSphere do the load balancing. When setting up my port groups, I usually just set them to route based on physical nic load.

Reply
0 Kudos
jsisen
Contributor
Contributor

Hi Enthusiat, 

My vm using vmxnet3. Max speed 10Gbs. On vss route base physic nic not support. Only support on vds.

how to method run max banwidth 

Thanks.

Reply
0 Kudos
Brisk
Enthusiast
Enthusiast

You can run multiple instances of iperf and see if that increases the throughput you're getting.

Reply
0 Kudos
jsisen
Contributor
Contributor

Ok i will test again and report

thanks

Reply
0 Kudos
Sachchidanand
Expert
Expert

Are you running your iperf server on the same host?

Regards,

Sachcidanand

Reply
0 Kudos
jsisen
Contributor
Contributor

yeap, I run iperf on the same host

Reply
0 Kudos
Sachchidanand
Expert
Expert

I can see you have the same subnet for iperf client and server means it's using the L2 enviornment for communication, you can try putting your client and server in to different vlan/network.

Also on VM it will show max bandhwith upto what you nic card support, in your case it''s vmxnet3.

Regards,

Sachchidanand

Reply
0 Kudos
jsisen
Contributor
Contributor

Hi All 

I try to test again with other vms on other server, still speed below do not improve. 

jsisen_0-1692170798036.png

 

thanks 

Reply
0 Kudos
Kinnison
Commander
Commander

Hello,


The point is that aggregating two network interfaces with speeds equal to 2 x 10 Gigabit/sec network interfaces does not at all equate to a grand total of 20 Gigabit/sec. There's a lot of literature out there, but there's a blog article well written that explains it pretty well, you can check it out here: https://frankdenneman.nl/2011/02/24/ip-hash-versus-lbt/


Regards,
Ferdinando

ansarabass
Enthusiast
Enthusiast

Hello Ahmad,

It seems like you're working on optimizing the network bandwidth between your ESXi host and Cisco switch using two 10G NICs in a port-channel configuration. Your goal is to achieve the maximum speed of 20Gbps between the host and the switch. Based on your description, it appears you're not getting the expected throughput.

Here are a few steps you can consider to troubleshoot and potentially improve the network performance:

NIC Configuration on ESXi:
Make sure that both 10G NICs are properly configured on the ESXi host. Check the speed and duplex settings to ensure they are set to the maximum capabilities of the NICs (e.g., 10Gbps, full duplex).

vSphere Configuration:
Verify that the NIC teaming and load balancing settings are correctly configured in your vSwitch or distributed switch settings. In this case, you're using the Route based on IP Hash algorithm, which should distribute traffic across both NICs based on source and destination IP addresses. This is a good choice for load balancing, especially for VMs communicating with multiple endpoints.

Cisco Switch Configuration:
It seems like you've already configured the port-channel on the Cisco switch. Double-check that the port-channel settings and load balancing configuration are consistent with what you've described. You're using the source-destination IP address load balancing method, which should help distribute the traffic effectively.

Physical Connectivity:
Ensure that the physical cables are properly connected and that there are no issues with the ports or cables. Sometimes, poor cabling can lead to lower-than-expected throughput.

MTU Settings:
Verify that the Maximum Transmission Unit (MTU) settings are consistent across the entire network path, including the ESXi host, vSwitch, Cisco switch, and any networking equipment in between. Mismatched MTU settings can lead to performance issues.

Network Traffic and Workloads:
Keep in mind that the actual network speed you achieve will depend on the type of traffic, the workload on the VMs, and the overall network utilization. If you're running tests with a single VM, you might not achieve the full 20Gbps speed due to various factors.

Network Testing:
Consider using multiple tools to test network performance, not just iperf. Try using ESXi's built-in performance monitoring tools or other third-party benchmarking tools to get a more comprehensive view of the network performance.

Firmware and Driver Updates:
Ensure that both your ESXi host and the Cisco switch have up-to-date firmware and drivers. Sometimes, performance improvements are delivered through updates.

Reply
0 Kudos