VMware Horizon Community
Gravis14
Contributor
Contributor

Instant clone cp-parent and NUMA awareness

The information around exactly how many cp-parents (for instant clones) get created per host is vague. Based on my experimentation, I'm seeing a cp-parent is created per NUMA node (and confirmed with esxtop). This would make perfect sense to me as the desktops share memory, and why not have the cp-parent on the same NUMA node for optimal memory colocation. Two cp-parents per host would also agree with the number of NUMA nodes in my dual-CPU Intel system.

I'm hoping someone who actually knows the cp-parent creation internals can confirm this, as I have also tried this on an AMD system with 4 NUMA nodes and still only see 2 cp-parents per host! I feel this is likely an oversight in the implementation by not accounting for >2 CPU systems or the AMD EPYC architecture.

The result on the AMD with 4 NUMA is 2 cp-parents, one on NUMA node 0 and the other on NUMA node 1. There are desktops running on all four NUMA nodes. Those on nodes 2 and 3 will have to cross boundaries to talk to the cp-parent shared memory, which is not ideal. I would expect to see 4 cp-parents to be created, each on their own NUMA node to service their memory-shared desktops.

I've confirmed the odd behavior with esxtop output as well. I'm running ESXi 6.7, 13004448 and View 7.6 9823717.

Thanks

0 Kudos
3 Replies
Kishoreg5674
Enthusiast
Enthusiast

Number of parent VMs created during priming operation = Max(NumHost, NumDataStore)

If number of hosts > number of datastores, then each host will have exactly one parent VM.  Some datastores will have multiple parent VMs pointing to the same Replica.

If number of datastores > number of hosts, then each Replica will have exactly one parent VM pointing to it.  Some hosts will have multiple parent VMs, each pointing to a different Replica.

0 Kudos
sjesse
Leadership
Leadership

This is good information, but is this all that is taken into account when creating the parents? I've been looking for this information for a year as we run 15 small instant clone desktop pools in one cluster and have almost as many parents as we do clones.

0 Kudos
Gravis14
Contributor
Contributor

Thanks for the explanation Kishoreg5674. I typically don't run in a situation where I have Datastores > Host. I tried this and it created an additional cp-parent (I have 3 now on at least one of my hosts).

Back to my original theory involving NUMA, is it true that VMware will not take into account NUMA boundaries with instant clones? Wouldn't it be ideal to have the desktops and the cp-parent sharing memory within the same NUMA node? Perhaps this should be an optimization to consider.

0 Kudos