Cannot migrate between identical hosts in same cluster - CPUID level 0x1 'ecx' mismatch

[ChristianWickha]

I am having problems migrating some virtual machine between hosts that are identical. They are new HP BL 490G6 servers, with a pair of x5570 processors @ 2.93Ghz , and I have enabled EVC mode "Intel Xeon 45nm Core 2", as there are also two other hosts in the cluster (HP BL480 G1 with a pair of E5430 processors @ 2.66Ghz).

I have ESX4, vCenter4 and all the latest firmware on all devices.

When I try to migrate the machines from one identical host to another, or to the other hosts, I get the following error;

Host CPU is incompatible with the virtual machine's requirements at CPUID level 0x1 register 'ecx'

Host bits: 0000:0000:0000:0000:0010:0010:0000:0001

Required: 1000:0000:0000:000x:xxx0:0x1x:xxx0:x001

Mismatch detected for these features:

• General incompatibilities; refer to KB article 1993 for possible solutions.

I consulted KB Article , and the sub-referred document

All the affected machines are VM Hardware 7, have the latest VMTools installed, have no RDMs or non-standard hardware, there are no floppy or CD drive mapped

I have tried editing the CPUID mask as specified in document 1008315, but this did not help. I also tried editing the CPUID mask for ecx to be 0--- -

---

-

---

-

---

-

---

-

---

-

---

-

---

and this did not help.

I powered down a VM to ensure that it had picked up the EVC settings. I am able to vMotion the VM when it is powered off, but this does not help as we have several machines "stranded" on one host and cannot be moved by DRS or manually.

The VMs are Windows 2003 (R2, SP2, x32). vMotion worked fine before on all hosts. Other identical machines from the same template are vMotioning fine, and when we were on ESX 3.5u2 this was not an issue.

Can anyone advise me on how I can diagnose and resolve this?

[NTShad0w]

hi ChristianWickham,

I have quite similar problems, but without EVC enabled, generally in short check that:

- there is a probablility that your vm gust start on second servers with other CPUs... I know in theory EVC should take care of tht CPU differences, but some software can read real cpu masks and... use it

- there is a possibility that U have some different bios settings or just different bios parameters

- there is a possibility that your cpus are not exactly the same, for example different stepping

- there is a possibility that for some reason on some hosts EVC just dont wokr, hmm dont ask why, just think about it

i had such situation, I have 4 vm hosts, all with different CPU )

so I create a masks, and I found correct mask for 3 hosts... maybe 3,5 hosts !!! why 3,5? because of that I can start machine on any of my 3 hosts and they migrate fine to all 3 hosts, but it cant migrate to this 4th one... but when with the same mask I just start a machine on this 4th host... I can migrate for any of my 4 hosts in any way )

my 2 hosts are ESX 4 and to ones are older and are ESX 3.5

think about my example and good luck

Dawid Fusek

IT Security Consultant &

Virtual Infrastructure Designer

COMP SA

[mdavis6890]

Did you enable Intel-VT within the BIOS?

Did you enable No Execute Memory Protection within the BIOS? This option has as

many names as there are hardware vendors but all contain either X-Bit

or No Execute in the name.

Both of those things need to be turned on for EVC to work properly.

[ChristianWickha]

Thanks for your help Dawid, but these are things that I have already checked.

I can vMotion other VMs between these hosts, I used to be able to vMotion these same VMs between these same hosts before we wiped 3.5u2 and upgraded to ESX4.

Does anyone know of a way to check on things like stepping, BIOS parameters etc, without rebooting the server? Even though I purchased all the blades at the same time, and they are identical models, and I made identical changes, it would be good to find out what ESX is detecting/using?

[ChristianWickha]

I have EVC enabled, and I have Intel-VT enabled and the NX bit is set to enabled - these are on by default in the blade model that is designed for virtualisation. I also verified all the BIOS settings before I installed ESX 3.5u2 (and unless ESX4 installer has some way of changing this, the settings will still be correct).

Does anyone know of a way to get ESX to report what CPU settings it is detecting?

Does anyone know why this is only affecting around 10% of my VMs?

[mdavis6890]

If it's only affecting a few of the VMs on that host, maybe it's a problem with the VM itself. If you can stand a little dowtime on that VM, try converting it to a template, and then cloning a new copy of it.

Michael

[Paul11 · Accepted solution]

I have had the same problem after the migration from ESX 3.5 to V4.0 with nearly the same HW-Configuration than you have. (BL490c and BL480c). But in my environment the articel you mentioned brought the solution:

To manually remove the CPUID masks using the VI Client:

1.Power Off the virtual machine.

2.Right-click the virtual machine and click Edit Settings.

3.Click the Options tab.

4.Select CPUID Mask under Advanced.

5.Click Advanced.

6.Click Reset All to Default.

7.Click OK.

8.Click OK.

9.Power on the virtual machine and migrate.

Have you really tried this?? The default mask is O.K., there is no problem with it in your environment, as i said already, I have the same hardware than you.

Paul

[beckhamk]

you didnt happen to upgrade the virtual hardware on the windows 2003 vms did you? We did and vmotion stopped only for our upgraded win2003 vms. There is a vmware kb that explains how to reset the cpu mask and then vmotion works again!

[ChristianWickha]

Perfect! This worked.

Very strange that some VMs were affected and others were not, and I have never changed the CPUID mask for any VM in the past. These VMs have had no need to change the CPUID mask as all my hardware was identical.

Thanks to Paul1:

To manually remove the CPUID masks using the VI Client:

1.Power Off the virtual machine.

2.Right-click the virtual machine and click Edit Settings.

3.Click the Options tab.

4.Select CPUID Mask under Advanced.

5.Click Advanced.

6.Click Reset All to Default.

7.Click OK.

8.Click OK.

9.Power on the virtual machine and migrate.