VMware Cloud Community
Edd_Barahona
Contributor
Contributor
Jump to solution

How to configure Memory to avoid NUMA problems on 4 proc server?

I have to design a ESX 3.0.1 to be installed on a 4 processor (AMD dual core). I have read that NUMA is to be cosidered to avoid posible issues.

In a 2 processors system is easy, just adding same amount of memory to each processor side, but for a 4 processor system, is it mandatory to have all 4 processors to have same amount of RAM? (cross balance) or is just necesary to have same amount of RAM in each pair? In other words, do I have to consider, for example, Processor 1 to have same amount of RAM tha Processor 3 o 4, or other crossed design?

My system is a HP Blade Bl45p. (2 processors one one side of the server, and 2 more at the reverse of the server).

Thanks

0 Kudos
1 Solution

Accepted Solutions
oreeh
Immortal
Immortal
Jump to solution

In a (cc)NUMA SMP system the memory is split across the CPUs.

If you have a ESX box with 12GB and 8GB "belong" to CPU0 and 4GB belong to CPU1 and you have a VM which uses 6GB memory

you will get performance penalties when the CPU scheduler uses CPU1 since this CPU only has fast access to 4GB RAM.

Therefore you should balance the RAM across all nodes in a NUMA system.

View solution in original post

0 Kudos
3 Replies
RParker
Immortal
Immortal
Jump to solution

What does NUMA stand for?

NUMA stands for Non-Uniform Memory Access.

OK, So what does Non-Uniform Memory Access really mean to me?

Non-Uniform Memory Access means that it will take longer to access some regions of memory than others. This is due to the fact that some regions of memory are on physically different busses from other regions. For a more visual description, please refer to the section on NUMA architeture implementations. Also, see the real-world analogy for the NUMA architecture. This can result in some programs that are not NUMA-aware performing poorly. It also introduces the concept of local and remote memory.

What is the difference between NUMA and SMP?

The NUMA architecture was designed to surpass the scalability limits of the SMP architecture. With SMP, which stands for Symmetric Multi-Processing, all memory access are posted to the same shared memory bus. This works fine for a relatively small number of CPUs, but the problem with the shared bus appears when you have dozens, even hundreds, of CPUs competing for access to the shared memory bus. NUMA alleviates these bottlenecks by limiting the number of CPUs on any one memory bus, and connecting the various nodes by means of a high speed interconnect.

What is the difference between NUMA and ccNUMA?

The difference is almost nonexistent at this point. ccNUMA stands for Cache-Coherent NUMA, but NUMA and ccNUMA have really come to be synonymous. The applications for non-cache coherent NUMA machines are almost non-existent, and they are a real pain to program for, so unless specifically stated otherwise, NUMA actually means ccNUMA.

The RAM isn't applied per CPU it's part of the entire system. So whatever amount you have available will be made available to the VM's. This should not a be a problem with AMD.

oreeh
Immortal
Immortal
Jump to solution

In a (cc)NUMA SMP system the memory is split across the CPUs.

If you have a ESX box with 12GB and 8GB "belong" to CPU0 and 4GB belong to CPU1 and you have a VM which uses 6GB memory

you will get performance penalties when the CPU scheduler uses CPU1 since this CPU only has fast access to 4GB RAM.

Therefore you should balance the RAM across all nodes in a NUMA system.

0 Kudos
Edd_Barahona
Contributor
Contributor
Jump to solution

Thanks for your Help, to avoid posible issues, I will use same amount of RAM for each CPU.

0 Kudos