How to create VM with same CPU cache topology as physical socket

[DyJohnnY]

Hi,

I am trying to make a comparison between running a workload on a physical host and running it inside a VM with the same specs as the physical host.

The workload is CPU intensive and leverages CPU cache a lot.

Running 7.0U2 as ESXi

The OS on the bare metal and the VM is Ubuntu 20.04

Hyper threading is on in both scenarios.

The lscpu output on physical host is:

lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Stepping: 6
CPU MHz: 1950.442
CPU max MHz: 3900.0000
CPU min MHz: 1000.0000
BogoMIPS: 4600.00
Virtualization: VT-x
L1d cache: 1 MiB
L1i cache: 1 MiB
L2 cache: 32 MiB
L3 cache: 44 MiB
NUMA node0 CPU(s): 0-15,32-47
NUMA node1 CPU(s): 16-31,48-63

The ouptut on the VM configured with 32 vCPUs is:

lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 43 bits physical, 48 bits virtual
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 1
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Stepping: 6
CPU MHz: 2294.609
BogoMIPS: 4589.21
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 64 KiB
L1i cache: 64 KiB
L2 cache: 2 MiB
L3 cache: 44 MiB
NUMA node0 CPU(s): 0-15
NUMA node1 CPU(s): 16-31

Is there a way to configure a VM in such a way that it will present the same amount of L2 cache to the guest OS, as the physical layout?

Thanks

[bluefirestorm]

This is a paragraph from the "man lscpu" page from an Ubuntu 20.04 desktop.

---

---

In virtualized environments, the CPU architecture information displayed
reflects the configuration of the guest operating system which is typi‐
cally different from the physical (host) system. On architectures that
support retrieving physical topology information, lscpu also displays
the number of physical sockets, chips, cores in the host system.

My plain English interpretation of that paragraph is "The lscpu information displayed in a virtualised environment is unreliable/incorrect".

Just to show how unreliable it is, if I configure 4 sockets, 1 core per socket for an Ubuntu 20.04 desktop VM (albeit in Workstation Pro not ESXi) with an i7-8700K (it has 12MB L3 cache) on the host machine, the lscpu in the VM shows 48MB L3. I don't think I can expect a bump in performance in the VM for something that does not exist (i.e the additional 36MB of L3 cache).

In any case, I don't think applications would be manipulating the L1/L2/L3 cache directly (I don't think it even can). It is primarily managed by the CPU itself although there are instructions to clear cache entries (such as INVPCID) but these are generally done by the OS CPU scheduler and won't be called by an application. That's why a bug like L1TF is a CPU bug and not an OS bug as the cache pretty much controlled by the CPU.