Good ol' vNUMA topic. As we create larger and larger VM's to virtualize larger workloads this topic comes into play. I've been watching this thread one and off for a while but still have the following question. Virtual Machine vCPU and vNUMA Rightsizing - Rules of Thumb - VMware VROOM! Blog - VMware Blogs
We are a fairly large shop ~5000 VM's across hundreds of hosts and a half dozen vCenters. One thing we instituted some time ago for uptime and convenience was hot-plug CPU/RAM on our VM's and templates. The vast majority of our VM's are 8vCPU or under so VNUMA being disabled is not a worry for those VM's.
My understanding was that if vNUMA was disabled via CPU hot plug only 1 UMA (not NUMA) node would be created regardless of vCPU count. I'm finding that via esxtop memory stats with NUMA stats enabled, all VM's that have more vCPU's than a single physical NUMA node, 2 UMA nodes are created (never more than 2 even on 4 socket hosts).
What we're seeing so far is that since at least 2 UMA nodes are created our remote memory access is fairly minimal. We do intend to load up these hosts with even more larger VM's so I'm assuming that with vNUMA being disabled we'll start to see performance degredation.
This is output from esxtop from a 4 socket 10core per socket host. The top VM named 24vCPU (24 1core sockets) shows UMA running on sockets 1 and 2. It shows 8Gig of RAM being accessed remotely under NRMEM. I'm assuming if we disabled hot-plug CPU and changed it to 3 socket 8 cores per the VM would start with 3 NUMA nodes and no remote mem access?
Thanks for any insight.
Ron
