Optimal CPU Setup for RDS-Servers on AMD Epyc

[ROBEL_IT]

Hello,

first off: I`m new here, hello everyone 🙂

Since we got our new DELL PowerEdge R7525 with 2 AMD EPYC 74F3 24-Core Processor i am trying to wrap my head around the best CPU-Setup for your needs in this environment.

My question is simple and i think since the paramenters are well known and do not change, someone can hopefully answer my question 🙂

But first, our setup:

Server: DELL PowerEdge R7525 with 2 Sockets
CPU: 2x AMD EPYC 74F3 24-Core Processor | 96 Logical Cores
RAM: 512GB
GPU: 2x nVidia A16
ESXi: 7.0.3 Build 20328353

Since we do not use nVidias GRID technology on the hypervirsor but pass through the GPUs as PCI-devices (8 in sum) we will have no more than 8 VMs on this host. Therefore our initial VM Setup looked like this:

10 vCPUs (10 Cores, 1 Socket) (in the attached Screenshots i reduced to 8 vCPU per VM)
62 GB of RAM fully allocated
PCI-Device of a GPU Attached
EFI Boot
Advanced Params:

pciPassthru.use64bitMMIO	TRUE
pciPassthru.64bitMMIOSizeGB	128

The VMs are booting a Windows Server 2016 of a vDisk from Citrix Provisioning (Upgrade to Server 2019 is comming soon)

In the BIOS of the DELL the relevat settings are these (default):

NUMA Nodes per Socket: 1
Number of CCDs per Processor: All
Number of Cores per CCD: All

As we got the VMs running on the surface everythings looks quite fine, but looking in esxtop I catched a feeling that something might be a bit off and could be better.
Specifically:

1. Since "only" 80 vCPU are assigned i would not expect %RDY times of sometimes around 2.0 - 3.0 (see attachment)
2. NUMA: We see quite much remote NUMA access, i would excpect 100% home node use (see attachment)

So what is the Ideal BIOS and VM Setup to get the best performance, considering that vMotion, DRS and so on is all turned of because of the attached PCI-Devices?

I already read through this article where it states that since 7.0.2 everything is handled by vmware using EPYCs

https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/performance/vsphere70u2-...

I hope i got all information needed in here and somebody can take a quick look at this setup.

Thanks in advance, Timo