VMware Networking Community
huchord
Enthusiast
Enthusiast

NSX-T VDS Uplinks with NUMA, performance less on Numa affinity 1

Have two identical VMs (one cloned from the other).  One VM has numa affinity for numa0 and the other VM for numa1.

Built a VDS within vCenter and pulled it into a Host Node in NSX-T.

Send traffic to the numa0 VM and we get a benchline of performance.

Send the same traffic to the numa1 VM and we get 10-20% less throughput.

Have checked all of the esxtop statistics and there's no indication of any vCPUs nor RAM being used "cross-numa".

Suspect that the vmnics are not aligned properly somehow.

 

[root] vsish -e get /net/pNics/vmnic4/properties | grep 'Device NUMA Node'

   Device NUMA Node:1

[root] vsish -e get /net/pNics/vmnic8/properties | grep 'Device NUMA Node'

 

 

   Device NUMA Node:0

 

screenshot from vCenter:

huchord_1-1690230338602.png

 

from esxcfg-vswitch_-l.txt:

huchord_2-1690230402000.png

This is from NSX-T:

 

huchord_3-1690230498461.png

 

Looking at this (this is just a guide, not architecture from our system) :

huchord_5-1690230687870.png

 

1) How can I check that the QPI bus (or whatever it's called for the Xeon 8358) was traversed?

 

2) How can I check that everything is vertically assigned?

 

TIA

 

 

 

 

Labels (2)
Tags (1)
Reply
0 Kudos
0 Replies