best processor for ESX server ?

[damiri]

Hello

I beleive there are tons of documents from Intel and AMD about who is better for virtualization purpose ( I know other components plays their part but here I would like to focus on processors). However, I would like to hear comments from this communitiy here since I know you have real life expeirence with this. Our customer is picking up HP c-3000 bladesystem and we have wide range of blades to use with XEON/Itanium/AMD Opteron processor support. Also I thing two-processor blade is enough for # of VM they are planning to run on each blade. It only matter of getting most for the money. So let me know your recommendations ...

Best regards,

damiri

[dkfbp]

Hi,

When we three months ago went for a HP C7000 our choice was BL460c with two Intel E5445 Quad core cpus.

We have been pretty happy with that choice.

Best regards

Frank Brix Pedersen

[meistermn]

1. Look at the vmmark results http://www.vmware.com/products/vmmark/results.html

2. It depends on the workload which processor is better: AMD (Shanghai) or Intel (Harperton 2 Socket or Dunnington 4 Socket).

3. Can you wait till April 2009 . Then Intel Nehalem is available.

[celak]

AMD Opteron second-generation and third-generation server processors especially in NUMA configurations.

Please see AMD's Direct Connect Architecture, HyperTransport and AMD Virtualization (AMD-V Pacifica) technologies' papers.

We should also consider energy efficiency of the processors.

Happy new year.

[williambishop]

Agreed, nehalem will be a game-changer I think..

[RParker]

XEON/Itanium/AMD Opteron processor support.

Best is a very poor superlative, and it's loaded. Opinions are not facts. People buy stuff for different reasons.

Best for what? Price? Best for quality, Best for compatibility, best for memory performance, best for your environment? If you have 30 ESX servers and they are all Intel, it's not a good idea to go out and buy 20 ESX servers equipped with AMD. So you have to qualify 'best'.

The industry best is Intel, only because most software programs are optimized for Intel processors and AMD are CLONES. AMD has a better price point, and tend to do better for memory performance, because of the onboard cache, but even that depends on what you are doing.

There is no 'best' for everything, Intel and AMD have their strengths and Weaknesses.

[damiri]

Best perfomance. It is going to be MSSQL, Exchange, file/printer load. Simple explanation, no rocket science. Better question is what would you buy if money is not a problem and your are limited to HP c-3000 bladesystem :-)?

damiri

[damiri]

We are thinking of bl495c as well as for bl460c ...

[meistermn]

Look at this thread and read the vmware world docu , which is mentioned.

http://communities.vmware.com/message/1135035#1135035

[WillGreen]

I'm also currently planning a blade purchase for VMWare and the BL 495s do seem attractive: they support 128 GB of RAM (or 64GB using cheap 4GB DIMMs) and the Shanghai Opterons are currently leading the VMark results. Anandtech have a good article on current server CPUs for virtualization here: http://it.anandtech.com/IT/showdoc.aspx?i=3484&p=9

Nehalem Xeons (55xx) will likely surpass the Shanghai Opterons in performance, but they're not here yet. Once the CPUs ship it will take time for robust chipset and VMWare support to arrive. Unless you're not buying until the second half of 2009 I wouldn't consider Nehalem. Furthermore, I imagine Nehalem Xeons won't be VMotion compatible with the current Core Xeons (54xx) - though I'm happy to be corrected on this.

[meistermn]

It is not only the processor. It is whole system.

1. What about Chipset?

IBM eX4 versus all other Vendors DELL, HP, SUN

Page 44-45 eX4 Chipset

http://www.azlan.ch/src/data/pdf/x3850_M2_6-Core_Xeon_MP_Proz_Customer_Pres.pdf

2. Dropdown of DRAM Speeds in AMD

IBM eX4 versus all other Vendors DELL, HP, SUN

http://www-03.ibm.com/systems/migratetoibm/systems/x/compare2.html

"The current implementation of the AMD™ Opteron™ processor supports DRAM

speeds of up to 667MHz. Due to the way the Opteron processor is designed,

however, when more than four DIMM slots are populated per processor, the speed

of all system memory drops to only 533MHz. The IBM System x3455 and x3755 avoid

this limitation through an IBM patented feature called IBM Xcelerated Memory

Technology™. Because of this innovation, all 64GB of RAM in the x3455 and x3755

run at the full 667MHz, using inexpensive industry-standard DIMMs."

This applies although to intel systems!!!

Different Response Times between 2 socket and 4 Socket Systems on vmware 3.5 exchange workload performance

2 Socket	300-400 ms	Figure 4 Page 3	http://www.principledtechnologies.com/Clients/Reports/Dell/PE2950virtualizedExchange08.pdf
4 Socket	650-700 ms	Figure 4 Page 3	http://www.principledtechnologies.com/Clients/Reports/Dell/PER900virtualizedExchange08.pdf

The response times are for 4 socket double high as for 2 socket systems? This strange ?

[damiri]

I think we will go with bl495c and flexnet on c-3000. I dropped IBM back in March 2006 (until then we were 10 years IBM business partner) and since then we are happy with HP, support, perfomance and price. Purchase will be done by end of January 2009 so I guess this is best possible option I can go with now... Thank you for your input here ...

damiri

[damiri]

IBM is not an option for me. Even do this is just a lot of marketing. I beleive what you have expeirenced with all aspect from puchasing to getting support from them :-)...

damiri

[WillGreen]

"The current implementation of the AMD™ Opteron™ processor supports DRAM speeds of up to 667MHz."

That document must be out of date. The current Shanghai Opterons (such as the 2384 and 8384) use PC2-6400 DDR2 SDRAM which runs at 800MHz.

[meistermn]

I am allthough on HP. But I will replace the second vendor through SUN, Dell or IBM.

I don't like IBM'S RSA Board. Dell Remote Board is OK. The new SUN i did not See.

The eX4 Chipset is not only marketing. It was a 100 million investment in this chipset. This chipset should reduce latency.

An other aspect is HP Virt ual Connect versus Xsigoor scalent or 3leaf

[damiri]

You are right about chipset. However, this is only small portion of the game, as you know. I made up my mind as I said. Will go with bl495c and flexnet interconnect. Thank you for your input here...

[smudger]

Hi - I just came across this post and thought I should relay some "experiences". We have been running HP hardware for many years and are extremely happy with it as a VI platform and for dedicated workloads.

Recently HP released the BL495c as a "specialist" VI blade, we took 2 of them on eval to test against our current BL460c (with E5450 chips). I have to say the BL495c in certain key hypervisor operations were up to 30% slower (Think CPU, Memory & Disk) than the BL460c's. I had lengthy dialogue with HP Houston and their testing team on our tests, BIOS settings etc etc you get the picture. The end result is that we will be sticking with the 460's for the time being.

To sum up, while I am a massive fan of HP hardware and technology, I think the marketing team behind the 495c got alittle carried away..........

Good luck,

Neil

[WillGreen]

Could you give more detail on what you mean by "key hypervisor operations"? Is this switching between guests and the like? Or when you say "Think CPU, Memory & Disk" do you mean that their performance was roughly 30% lower across the board? What sort of applications are you running? What speed were the Opterons you went for? Does your ESX support NPT (added in 3.5.1 I believe) and was NUMA configured?

The current "Shanghai" models seem to do very well for virtualization and for memory intensive workloads, though they're still beaten by the latest Xeons in some benchmarks. I'm surprised (and a bit concerned) that you're seeing 30% lower performance. See http://it.anandtech.com/IT/showdoc.aspx?i=3484 for a good comparison of the current server CPUs.

[damiri]

HM, I will takt this to our HP contact. I will take eval of the bl495c first just to avoid thigs like this ...

[smudger]

Hi - the tests were performed mostly by a third party tool called "Burnin Test Pro". The tool is available for free, we use this (and some inhouse tools) to benchmark all types of hardware to certify for our RHEL5, MS Server & ESX builds. I have attached a screenshot of the testing results. Each host, 495 & 460 were running 6x MS Server 2003 VMs. The performance from the 495 was significantly slower for memory operations, disk. This was clearly visable when viewing using esxtop

To answer your question on type of Opteron's and speed - we were using the 2.3Ghz, 2356 Quad Core vs 3Ghz E5450 Quad Core Xeon.

While obviously the clock speeds are different, the AMD architecture, especially memory should make up the differences or so we thought! I spent a number of days working with HP on ensuring AMD PowerNow features were disabled, NUMA was enabled/disabled, updated BIOS and various other settings. Some other interesting figures are:

Check out spec.org for public performance results, however:

The per-core spec_int for the AMD 2356 is 13.2 whilst the Intel 5450 is 23.1 so the Intel is 75% faster.

Overall spec_int_rate for the AMD 2356 is 89.5 whilst the Intel 5450 is 109 making the Intel 21% faster

To finalise the testing I rebuilt each blade with W2K3 Server and ran the same tests, thus removing the hypervisor layer/overhead. Similar results were passed which drew us to sticking with Intel and ultimately the BL460c. Again, this was backed up by our HP TAM and the Houston Performance Team. I may have time in Feb to look at the newer Shanghai models, this would be interesting.

If you guys can help shed some light on this then great, we all want a fast and consistant VI platform and ultimately to see the hypervisor "overhead" to shrink as much as possible.

Thanks,

Neil