Mixing AMD Opteron 200 series and 2000 series CPUs on adjacent ESX hosts

[evdcs]

I have ten HP DL385 G1 ESX hosts spread across four environments and need to increase server capacity. HP doesn't make the DL385 G1 and none of my vendors can get them. The boxes I have already have 16 GB each and adding more RAM will probably create a bottleneck at the processor.

For two environments, with two ESX hosts each, I want to add one more host - a DL385 G2 with Opteron 2218 processors. The existing hosts have Opteron 280 processors. We're also adding Equallogic iSCSI storage to these two environments.

I'd like to be able to use DRS and migrate the guests among three hosts. I understand the 2000-series processors will support more instruction sets, more features, than the 200 series. If I build a machine on the 2000 series Opteron, it will expect these features and won't move to a 200-series Opteron server without a conversion.

Is this a valid concern? If it is, how do I make ESX (or the server) ignore the new processor features on the Opteron 2218 so I can move VM's between servers?

[oreeh]

Is this a valid concern?

Yes this is a valid concern.

The only way to convince ESX/VC is to set CPUID masks.

Read KB article 1993[/url]

Be aware that setting these masks doesn't prevent the guest from detecting and using these features!

IMHO your best bet is to switch servers between the locations.

[evdcs]

You're probably right. This seems like a big drawback - how can I ever add new servers to an ESX server farm when different generations of the same processor create incompatible VM's?

I can shift servers around this time because I have two office-LAN servers which don't have to be identical to prod/dr/qa boxes. But the prod/dr/qa boxes need to match... I have some slightly older DL585 G1 servers which, when they're replaced, could go into the ESX pool, but they'll have the same problem, adding an older generation of processors to the existing environment.

There's no way to suppress processor features in ESX or Virtual Center?

[oreeh]

The VMs are not necessarily incompatible.

This depends on the software used on the VMs.

The Windows OS normally doesn't have a problem with a CPU change.

As long as the apps don't use these features you are fine with a vMotion mask.

With Linux this is a completely differents story.

The problem with the masks is that an application is still able to detect the CPU using the CPUID instruction.

And since the CPUID instruction is an unprivileged instruction there's no way to intercept it from ESX.

The real problem is the x86 architecture and not ESX!

The only way to prevent this would be to emulate instead of virtualize the CPU - which slows down the whole thing a lot.

Message was edited by:

oreeh

[BenConrad]

You are going to need to set the CPU mask on every single VM (ie, shutdown, edit, reboot) before you migrate to the G2 hosts.

We have DL385 G1, G2 and DL585 G1 & G2 in the same cluster/datacenter. I think we set 3-4 masks and we also turn off the NX bit.

[moberle]

This is not necessarily true. CPU masking can be set at the Virtual Center level by adding masking to the configuration xml file. This is not supported by VMWare but it works. There are several threads on this subject in the forums.

[BenConrad]

Hi Moberle. Can you provide a few links to these threads where people are running VC 2.x & ESX 3.x and have successfully modified vpxd.cfg so that new virtual machines inherit these settings?

I've read this post but it's short on results: http://www.vmware.com/community/thread.jspa?messageID=655717&#655717

From what I have see only ESX 2.x falls under this category:

http://kb.vmware.com/selfservice/viewContent.do?externalId=1993

Ben