VMware Communities: ESX NUMA error going from 3.5 to 4

Skip navigation

VMware

Up to Discussions in VMware ESX™ 4

This Question is Answered (go to answer)

5,461 Views 7 Replies Last post: Oct 27, 2009 11:02 PM by mudha

JPeluso

10 posts since
Apr 2, 2008

Currently Being Moderated

Jun 22, 2009 5:58 PM

ESX NUMA error going from 3.5 to 4

Hello Everyone,

I have a slight problem with all my ESX 4.0 hosts. After upgrading from ESX 3.5 to 4.0 the error below popped up.

Hardware:

HP C3000 Blade Chassis OA firmware 2.32

BL 465 G5 Bios Rev A13

2 X Quad AMD 2356

24 gbs of ram.

Error:

TSC: 1137350434cpu0 )NUMA : 827 Signficant imbalance between NUMA nodes detected. Performance may be impacted.

I tried searching the forums for anyone with the same issue but, I had no luck. Has anyone ever run into anything like this?

Thanks for the help,

James

Tags: hp, error, amd, tsc, numa, esx4, esx_4, performance_impact

pharmer

22 posts since
Jul 23, 2007

Currently Being Moderated

1. Jun 22, 2009 6:57 PM

in response to: JPeluso

Re: ESX NUMA error going from 3.5 to 4

You will need equal memory sizes per processor socket in your server.

I'm not familar with the Opteron setup, but with a 2 socket Nahalem box, the memory needs to be balanced eqaully in size per socket for NUMA to work correctly (or else this error occurs)

e.g. Socket 1, 16GB (4 x 4GB), socket 2, 16GB (4 x 4GB)

You may have to look at your server docs to see proper channel and memory configuration for a multi socket system

Report Abuse

JPeluso

10 posts since
Apr 2, 2008

Currently Being Moderated

2. Jun 22, 2009 7:11 PM

in response to: pharmer

Re: ESX NUMA error going from 3.5 to 4

Hey Pharmer,

Thanks for the FYI. I'm going to check out one of my blades tomorrow and verify the DIMM configuration is correct.

Jim

Report Abuse

BenConrad

710 posts since
Mar 20, 2006

Currently Being Moderated

3. Jun 23, 2009 9:24 AM

in response to: JPeluso

Re: ESX NUMA error going from 3.5 to 4

You should be able to see this in 'esxtop' -> 'M', there is a NUMA row that displays how much memory each NUMA node contains and how much of that memory is in use. Here is one of my servers, as you can see the memory is unbalanced but it's not too bad (14GB vs 16GB)

NUMA /MB: 13527 ( 5719), 16226 (12056)

Ben

Report Abuse

depping

4,243 posts since
Jan 17, 2005

Currently Being Moderated

4. Jun 23, 2009 10:07 AM

in response to: BenConrad

Re: ESX NUMA error going from 3.5 to 4

http://www.yellow-bricks.com/2007/12/28/memory-incorrectly-balanced/

Duncan

VMware Communities User Moderator | VCP | VCDX

Blogging: http://www.yellow-bricks.com

Twitter: http://www.twitter.com/depping

If you find this information useful, please award points for "correct" or "helpful".

Duncan | Yellow-Bricks.com | Author of the vSphere 5.0 Clustering Deepdive

Report Abuse

JPeluso

10 posts since
Apr 2, 2008

Currently Being Moderated

5. Jun 23, 2009 10:19 AM

in response to: depping

Re: ESX NUMA error going from 3.5 to 4

Ok so Everyone helped fix this with there suggestions.

ESXTOP gave me some troubleshooting information as well as the blog over at yellow-bricks.

The memory on the servers was not equally spaced out based on the NUMA for the seperate dual processors.

Thanks again everyone,

Jim

Report Abuse

sysmast

10 posts since
Apr 23, 2008

Currently Being Moderated

6. Oct 27, 2009 2:29 PM

in response to: BenConrad

Re: ESX NUMA error going from 3.5 to 4

OK newbie question, what is the "-> M" portion of the ESXTOP command....i've seen ESXTOP, but I'm feeling like i'm missing somethig to see the NUMA data. i've got the same error as the main post here.