Nehalem memory config suggestions

[BrennanB]

A Nehalem server was purchased before we understood the new memory structure . We received a Dell PowerEdge R710 with 2 x Xeon X5560 with the memory configured with Advanced ECC. It has 8 x 4GB Dual Rank dimms installed since it was ordered using the Advanced ECC configuration. After doing some research on how the new memory works I would appreciate some opinions on how best to proceed to chang ethis to performance.

I don't see a real need to run Advanced ECC and would think Optimized would be the preferable route to go. So would the best path be to remove current config and populate each of the 3 channels with just a single 4GB dual rank chip and then down the road we can buy 4 more chips to then populate 2 DIMMS in each channel? Or can we populate 2 of the 3 channels with 2 DIMMS each? IF this second method is possible will we take a large performance hit?

So basically

A1 - 4GB Dual Rank

A4

A7

A2 - 4GB Dual Rank

A5

A8

A3 - 4GB Dual Rank

A6

A9

or

A1 - 4GB Dual Rank

A4 - 4GB Dual Rank

A7

A2 - 4GB Dual Rank

A5 - 4GB Dual Rank

A8

A3

A6

A9

[JasonVmware]

Hey,

I had the same problem with the M710 series of servers recently and what I did until update 1 comes out (Release is supposed to happen between Sept-Nov this year) is turned off the large memory option in VMware till then. This switch is a simple change in the advanced configuration and can be put back shortly after update 1 is installed. I have not seen ANY performance issues with turning it off. Remeber this option wasn't even avilable in ESX 3.5 which begs the question how much performance are you really loosing seeing as 3.5 ran pretty good!

To turn off the memory option go into Configuration>Advanced>Mem> Mem.AllocGuestLargePage

Change this value to 0 and reset your esx host. After the hosts / vms came back online the memory usage went back to normal and has stayed that way since.

Refrence: http://communities.vmware.com/message/1329411

I also ran into an issue with the M710 series where the iDRAC BMC module was miss reporting or the reporting it did was read incorrectly by ESX4. This caused false CPU usage alarms in vCenter4. I talked with VMware and they said this was a Dell issue. I have escalated this to Dell and they have replicated the problem and feel it will be a firmware fix. The work around for this issue if you run into it is to turn off the IMPI alerting. This can be done by editing the 72.impi script under /ect/vmware/init/init.d if you need more information let me know as I don't have the exact line of code you have to enter into the script at the moment. I will probably be making a post about these issues to help the community in the near future. Hope this helps !

[beyondvm]

With the nehalem chipset it is preferred to keep an uniform memory distribution across all channels of memory (there are 3 now), meaning you need to upgrade your memory in groups of 3 DIMMs. Per the dell website, the R710 has 18 DIMM slots, so those are broken across the 3 channels. If you have 8 DIMMs in each server I would either take 2 out or buy one more 4gb dimm for each server so you would end up with:

A1 - 4GB Dual Rank DIMM

A2 - 4GB Dual Rank DIMM

A3 -

A4 -

A5 -

A6 -

B1 - 4GB Dual Rank DIMM

B2 - 4GB Dual Rank DIMM

B3 -

B4 -

B5 -

B6 -

C1 - 4GB Dual Rank DIMM

C2 - 4GB Dual Rank DIMM

C3 -

C4 -

C5 -

C6 -

With 6 4GB DIMM in each server OR:

A1 - 4GB Dual Rank DIMM

A2 - 4GB Dual Rank DIMM

A3 - 4GB Dual Rank DIMM

A4 -

A5 -

A6 -

B1 - 4GB Dual Rank DIMM

B2 - 4GB Dual Rank DIMM

B3 - 4GB Dual Rank DIMM

B4 -

B5 -

B6 -

C1 - 4GB Dual Rank DIMM

C2 - 4GB Dual Rank DIMM

C3 - 4GB Dual Rank DIMM

C4 -

C5 -

C6 -

With 9 4GB DIMM in each server. This is not required, but will give you optimal performance with the nehalem chipset. Hope that helps!

---

If you found any of my comments helpful please consider awarding points for "Correct" or "Helpful". Thanks!!!

www.beyondvm.com

[BrennanB]

Thanks

[beyondvm]

No problem, I have since written a post on my blog about memory with nehalem, hope that helps too.

---

If you found any of my comments helpful please consider awarding points for "Correct" or "Helpful". Thanks!!!

www.beyondvm.com