Gavster2002
Contributor
Contributor

Vmotion fails after vSpere upgrade, CPUID errors

Hi,

I have upgraded from ESX 3.5 to ESX 4.0 and now I am having issues with migrating hosts, I have found article 1011294 that describes my problem and it advised me to change the CPUID mask for all my VM's.

Is there a way to get round this without having to power off all 192 VM's I have? Seems like a lot of work doing it manually like this.

Thanks for any help

Gav

0 Kudos
9 Replies
mlubinski
Expert
Expert

I will probably have the same problem soon, so I would also like to hear tips from someone who already did solve this issue Smiley Happy

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

[I]If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points[/I]
0 Kudos
Troy_Clavell
Immortal
Immortal

it's a known issue, see this KB

http://kb.vmware.com/kb/1011294

0 Kudos
Gavster2002
Contributor
Contributor

My original post stated I have read this article, I am looking for a better way of doing it as I have nearly 200 vm's and doing it to every vm is very time consuming.

Thanks for your post tho, any thoughts on how to do this as a global setting or something?

Gav

0 Kudos
Troy_Clavell
Immortal
Immortal

sorry... Maybe I should have read the original post a little closer. Seeing that the VM must be powered down to reset CPUID bits, it will require a shutdown of each VM that is affected. Every single guest in your environment is experiencing this issue?

You can probably script the reset of the CPUID with PowerShell

0 Kudos
Gavster2002
Contributor
Contributor

It seems that about 50% of the VM's are complaining, either way it is a pain.

I have also found this article: 1008315 VMotion errors between different Intel 45nm Core 2 revisions, I have one of these processors so not sure if this is the problem?

Shame there is no central setting you can apply to allow the vm to ignore this constraint, oh well, guess I am working again this weekend! Smiley Sad

Gav

0 Kudos
dnetz
Hot Shot
Hot Shot

If i'm not mistaken, PowerCLI can change your CPUID masks (like so: http://ict-freak.nl/2009/08/19/powercli-set-cpu-identification-mask/) but it seems it's not as easy to script the "Reset All to Default" button in the vSphere client since that depends on both the host and guest OS i.e. you probably can't apply the same mask to all VM's. But if you're just out to mask one extension from all your VM's then you could certainly script it.

Edit: perhaps if you copied the current EVC masks (Cluster settings -> Vmware EVC -> Current CPUID Details...) and applied to all VM's via PowerCLI? Can't really test this so I can't say for sure.

adamy
Enthusiast
Enthusiast

Probably not exactly what you were hoping to hear, but I do not think it is possible to automate. I have attached the steps we use to prevent this from happening in the first place.

Upgrading VMware tools and Virtual Hardware to run on an ESX 4.0 Host.

  • 1) Power VM off.

  • 2) Right Click on the VM and choose Migrate. Verify that "Change both host and datastore" is selected. Click next

  • 3) Choose the appropriate Farm and any Host and choose next.

  • 4) Choose a SAN datastore. Preferably the one with the most available space

  • 5) Generally you will choose "Same format as source" option( It should be the choice on top) and click next.

  • 6) Review and choose Finish

  • 7) The copy will take some time and depends on the size of the VM being moved.

  • 😎 Once the move operation is complete power up the VM.

Important: Do not upgrade the Virtual Hardware without the latest tools installed. The system may become unrecoverable if you do!

  • 9) Once the VM is powered up you need to upgrade the VMware tools: Right click on the VM and choose Guest à Install/Upgrade tools. You can choose to manually install by logging in to a VM or you can choose the automatic process that installs and reboots a VM. If you choose do this manually (this is the best option) choose "typical" install.

  • 10) Once this is complete you will need to shutdown the VM

  • 11) Right Click on the VM and choose "Upgrade Virtual Hardware" and choose next to the warning.

  • 12) Once this is complete you will need to power up the system.

  • 13) Login to the system. New hardware will be installed. When this Is done you need to power the system off.

  • 14) Reset the CPUID. Once the VM is powered off Right Click and choose Edit Proprities.

  • 15) Go to the options Tab. Highlight the CPUID Mask option.

  • 16) To the Right click on Advanced. Click Reset All to Default. Click OK and OK again.

  • 17) Power the system up.

Hopefully these steps will prevent others from having this problem.

Adam

Gavster2002
Contributor
Contributor

Thanks for all the advice, I think a one off manual excersise is probable the easiest way to get round this.

Gav

0 Kudos
jjewett
Enthusiast
Enthusiast

This answer is unacceptable. When upgrading from ESX 3.5 to version 4.0 we shouldn't be required to Power Off every VM in order to Migrate. Granted, to completely upgrade every VM will need to be rebooted for the VM tools update and Powered Off for the Virtual Hardware upgrade. However upgrading all my ESX servers to 4.0 cannot be completed because VMs cannot be moved off the last ESX 3.5 servers.

How the CPUID mask got there in the first place is a mystery. All the blades we have are the same model and CPU type. Most were created from template on these very blades.

I do have a PowerCLI script that I painstaking crafted to clear all CPUID masking. Essentially it is the CLI equivalent to pressing Reset All Rows to Default. But this script doesn't help me because although it works and modifies the mask ( I've verified with a power off), the VM is still running with the old mask applied and a power off and on is the only thing that enables the VMotion compatibility again.

I feel burned by VMware because of this CPUID glitch. We make every effort to be thorough in our upgrade processes and try to use VMware's technology to its fullest. But issues like this make me disallusioned by VMware's promises of improved uptime and reliability through the upgrade process. I'm surprised that companies with hundreds of VMs aren't more upset to have to manually reset the CPUID mask on all their systems.

Sorry for venting.

Jonathan

0 Kudos