VMware Cloud Community
KNardi
Contributor
Contributor

Vmotion issue

I am running ESX 3.5 build # 143128 on 2 Proliant BL465c G5, 8CPU quad core opteron's, I also have 1 Proliant BL 465c G1 4cpu, dual core opteron.

All 3 are in the same cluster.

I can vmotion between the BL465c G5's, but when I try to vmotion to the 465c G1 I get this error: "Unable to migrate from xxxx to xxxx: Host CPU is incompatible with the virtual machine's requirements at CPUID level 0x80000001 register 'ecx'. host bits 0000:0000:0000:0000:00000:0000:0000:1001 required 0000:0000:0000:0000:00xx:0xx1:111x:xxxx

refer to KB article 1993 for possible solutions.

I thought when going to ESX 3.5 update 2 that you could cross hardware and still use VMOTION??

Please advise!

0 Kudos
8 Replies
admin
Immortal
Immortal

The problem is the CPU features are different between source and target host, as the CPUs are passed almost directly to the guests suddenly changing what the CPU is while the OS is running could well cause the guest OS to crash - which is why VMware is refusing to do it. You need to apply a CPU mask to the VMs to match the CPU feature set of the lowest common denominator - i.e. your dual core CPU, this means when running on the quad cores the VMs will not get to use the features that are not available on the dual core but you will be able to VMotion from between the different CPUs. The ESX version is largely irrelevant in this case.

More info here: http://kb.vmware.com/selfservice/viewContent.do?externalId=1993

Further reading a a tool to help identify CPU features here: http://blog.scottlowe.org/2006/11/23/vmotion-compatibility/






www.phdvirtual.com

0 Kudos
KNardi
Contributor
Contributor

Ok....I was recently at a VMUG seminar in Boston, one of the AMD reps gave a presentation saying that ESX 3.5 update 2 corrected this problem. As long as the hardware was AMD and on the compatibility list.

Was he just blowing smoke?

0 Kudos
TCronin
Expert
Expert

In order to make it work you have to enable EVC on the cluster. To do this though you have to have no powered on guests on the hosts. What I did was make a new cluster with EVC enabled (Enhanced Vmotion Compability) and put one host in maintenance mode and move it from the existing cluster to the new cluster. I then moved all the guests off of another host in the old cluster to the new cluster using vmotion, put the empty one in maintenance mode and move it to the new cluster. Repeat until done.

I know it's a lot of extra work, but it is well worth it for the ability to move between version of hardware.

Tom Cronin, VCP, VMware vExpert 2009 - Co-Leader Buffalo, NY VMUG

Tom Cronin, VCP, VMware vExpert 2009 - 2021, Co-Leader Buffalo, NY VMUG
0 Kudos
admin
Immortal
Immortal

Ooh new feature. My bad, but the principle is the same. Guess it just auto-applies the appropriate mask to the VMs then? Do the CPUs do the actual feature mask or just report to VMware what the mask needs to be?




www.phdvirtual.com

0 Kudos
KNardi
Contributor
Contributor

This is awesome, I am waiting for one client to shut down before trying this. The extra work is nothing compared to the alterenative...

I'll give you the YES, my quesion has been answered as soon as I test.

thanks so much!

PS. Why wouldn't that feature be enabled by default?

0 Kudos
TCronin
Expert
Expert

With it enabled it changes the mask to default the the lowest common denominator. For AMD that is Opteron Second generation. If you have software that requires one of the newer instruction sets (haven't actually come across that yet) then it would break.

I think they want to be sure you really intend to implement it.

I forgot in the other post, I think I had to put my newest hosts in first to make vmotion work between the clusters. That way the newer instruction sets were masked out and matched the guests on the older hosts.

Tom Cronin, VCP, VMware vExpert 2009 - Co-Leader Buffalo, NY VMUG

Tom Cronin, VCP, VMware vExpert 2009 - 2021, Co-Leader Buffalo, NY VMUG
0 Kudos
KNardi
Contributor
Contributor

I created another cluster, put my ESX server in maintenance mode and try adding it to my new cluster, now I am getting an error saying:

"a general system error occured: The hosts CPU hardware should support the Enhanced VMotion Compatibility mode that is enabled for this cluster, but some of the necessary CPOU features are missing. Chekc the hosts BIOS configuration to ensure that no necassary features are disabled (such as XD and VT for Intel, or NX for AMD)"

0 Kudos
KNardi
Contributor
Contributor

I finally found a fix for this.

In addition to adding EVC to the Cluster, I also had to go into the HOSTS BIOS. Processor/Advanced options and enable AMD Virtualization & also enable No Execute Page Protection. Then poer off the HOSTS.

After they powered up I was able to add them to the new cluster. Then I had to disable HA/DRS on the cluster and re-enable them. Of course taking the HOSTS out of maintenance mode.

Now all is good....FOR NOW!

Thanks to all that helped

0 Kudos