VMware Cloud Community
JPeluso
Contributor
Contributor
Jump to solution

ESX NUMA error going from 3.5 to 4

Hello Everyone,

I have a slight problem with all my ESX 4.0 hosts. After upgrading from ESX 3.5 to 4.0 the error below popped up.

Hardware:

HP C3000 Blade Chassis OA firmware 2.32

BL 465 G5 Bios Rev A13

2 X Quad AMD 2356

24 gbs of ram.

Error:

TSC: 1137350434cpu0 :smileyshocked:)NUMA : 827 Signficant imbalance between NUMA nodes detected. Performance may be impacted.

I tried searching the forums for anyone with the same issue but, I had no luck. Has anyone ever run into anything like this?

Thanks for the help,

James

Reply
0 Kudos
1 Solution

Accepted Solutions
depping
Leadership
Leadership
Jump to solution

http://www.yellow-bricks.com/2007/12/28/memory-incorrectly-balanced/

Duncan

VMware Communities User Moderator | VCP | VCDX

-


Blogging: http://www.yellow-bricks.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

View solution in original post

Reply
0 Kudos
7 Replies
pharmer
Enthusiast
Enthusiast
Jump to solution

You will need equal memory sizes per processor socket in your server.

I'm not familar with the Opteron setup, but with a 2 socket Nahalem box, the memory needs to be balanced eqaully in size per socket for NUMA to work correctly (or else this error occurs)

e.g. Socket 1, 16GB (4 x 4GB), socket 2, 16GB (4 x 4GB)

You may have to look at your server docs to see proper channel and memory configuration for a multi socket system

JPeluso
Contributor
Contributor
Jump to solution

Hey Pharmer,

Thanks for the FYI. I'm going to check out one of my blades tomorrow and verify the DIMM configuration is correct.

Jim

Reply
0 Kudos
BenConrad
Expert
Expert
Jump to solution

You should be able to see this in 'esxtop' -> 'M', there is a NUMA row that displays how much memory each NUMA node contains and how much of that memory is in use. Here is one of my servers, as you can see the memory is unbalanced but it's not too bad (14GB vs 16GB)

NUMA /MB: 13527 ( 5719), 16226 (12056)

Ben

depping
Leadership
Leadership
Jump to solution

http://www.yellow-bricks.com/2007/12/28/memory-incorrectly-balanced/

Duncan

VMware Communities User Moderator | VCP | VCDX

-


Blogging: http://www.yellow-bricks.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Reply
0 Kudos
JPeluso
Contributor
Contributor
Jump to solution

Ok so Everyone helped fix this with there suggestions.

ESXTOP gave me some troubleshooting information as well as the blog over at yellow-bricks.

The memory on the servers was not equally spaced out based on the NUMA for the seperate dual processors.

Thanks again everyone,

Jim

Reply
0 Kudos
sysmast
Contributor
Contributor
Jump to solution

OK newbie question, what is the "-> M" portion of the ESXTOP command....i've seen ESXTOP, but I'm feeling like i'm missing somethig to see the NUMA data. i've got the same error as the main post here.

Reply
0 Kudos
mudha
Hot Shot
Hot Shot
Jump to solution

run esxtop command and then type m for memory there are various option availabkle eith this search for help

Reply
0 Kudos