Can you find out if we can use vmotion on an ESX server with 2 single processors with hyperthreading (3.6 Ghz) and an ESX server with dual processors (2.88 Ghz)?
There is a legend in the VI Client which explains the different characters.
VM -> Edit Settings -> Options -> Advanced -> Advanced -> Legend
I guess I need to open my eyes :smileygrin:
Thanks.
For example, here is what VC 2.0 creatd during cold
migrations:
"--
xxxxxxxx----xxxx------"cpuid.1.ecx = "--------------
R-0--R----"cpuid.1.ecx.amd = "--------------------------------"
cpuid.80000001.ecx =
"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0"
cpuid.80000001.edx =
Ah - this could be most useful to me - I'm having no luck migrating from Opteron 200 series (HP BL35p) to Opteron 800 (HP BL45p)
Where does VC2 write this information during a cold migrate?
ta
tobi
It sticks it in the vmx file.
Guys, whilst I was at VMworld last week I attended a session entitled:
"Migrating between Apples and Oranges with VMware VMotion in VMware Infrastructure 3"
http://www.rtfm-ed.co.uk/?p=309
I was a good session - but I felt it still left some questions unanswered. After the session I was approached by some delegates asking if "RTFM Education" would be writting a whitepaper on the topic.
This is something I would like to do, but I feel quite out of my depth. Also I think such an undertaking would require very[/i] rigious hardware testing. Something that one-man-shop like me would struggle to do properly - through a lack of equipment. In fact I think in general such a whitepaper could only be[/i] created by a joint community effort.
So that's why I am adding this post today - to propose something. In the abscence of any "idiots" guide or cook-book on using the advanced CPU masking feature, I propose that we as a Community produce our own whitepaper. I wouldn't take any credit for the work - that would go to you guys who contributed (either in an authorial or testing manner). I would be happy to host the whitepaper on my website - and hopefully that would help disseminate this knowledge to the wider community. It would join the library of free guides and documents I host at RTFM Education already.
This is the way I see my site going. As I become more busy with paid-for-work - the amount I give back to community is under-pressure. I would like to do more - but time is not endless resource - and the book I am writing with Ron & Scott must take priority in the short/medium term. I would however, like to kick-start some Community driven projects - using my website as a platform to get them out to the Community at large. Remember my site has completely free content - and contains no ads - so its about as "neutral" and not-for-profit a space there can be that still gets a reasonable amount of hits. I would hope this project would not go unnoticed - and would benefit the reputation of those who choose to contribute.
Anyway. If this is particular topic is off interest to you please respond. I think once the "Apples & Oranges" PPT's are released this will provide a good framework for documenting the CPU mask feature. There was a few tidbits in there which I thought the community at large would benefit from.
Kind Regards
Mike Laverick
RTFM Education
oh i should add - something i hadn't noticed - that KB1993 has been updated to add some addition info about advanced CPU masks - I imagine you guys knew this already - but it had slipped under my radar...
http://www.vmware.com/support/kb/enduser/std_adp.php?p_faqid=1993
Regards
Mike
Hendrick, becareful masking "all" new features. I was at the session Mike spoke of and the best tidbit I walked away with is the fact that ESX performas Binary translation on the OS only. Not the application execution. This makes sense (now) as the application will run at ring level 3 and can't make illegal call on the CPU. What it does means though is that the masks that you create do not apply to the application. VMotioning an application from a CPU with SSSD to one with out could cause your application to do something bad if it calls that instruction set. VMware supports masking the NX bit but everything else you are on your own for.
If I mask all the new features per VM , it should not make a difference if I am running it on DL380G4 or on a DL580G2. Or am I wrong here ? The application is not expecting the new feature , so it is not getting the new features on either system (masked on both systems).
I have been using the following masks for some months now ... didn't have any problems with them yet...
Level 1
eax -
XXXX XXXX -
-
XXXX -
-
ecx -
-
-
-
R-0- -
0R-0 00-0
Level 80000001
ecx XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXX0
edx XX0X XXXX XXX0 XXXX XXXX XXXX XXXX XXXX
This is precisely the kind of information - that needs putting into a single white paper on the issue & tools...
Keith is right on the money on this one...
Do we have any takers to write a whitepaper... I've already had Scott Lowe of Jumpbox express an interested. He's taken stuff from this thread and blogged about it. But I think we all know it needs cooking down - drawing all these tidbits in a single document that become the kind of "bible" on this topic generally... the information/knowledge is out there in the community it just needs consolidating & co-ordinating... if we don't do it - who will???
Regards
Mike
Hendrik, you are correct with regard to the OS but what we learned at VMworld is that the applications are not masked. That is it can "see" the CPUs true nature despite the VMware mask. I don't expect many applications to have a problem, most will never directly use SSSD instructions and their for never have a problem. The problem is we don't know which ones do or when they might decide to call them.
Mike, I am definitely willing to help however I can. Count me in.
Does anyone has a mask for these two processor types ?
XEON 5030 (SSE3) and XEON 5160 (SSE4)
I read the VMware Docs 1991 and 1993 to ignore SSE4 instructions (and also SSE3 combinded with SSE4) but i get everytime the same error when i want to migrate or use vmotion:
unable to migrate from xyz to zyx:
The CPU of the host is incompatible with CPU feature requirements of virtual machine; problem detected at CPUID level 0x1 register 'eax'
Servers are Dell 2950, same hardware except of the XEON CPUs
I´ve got another information about migrating between ESX Hosts.
If Intel VT is activated on only one of the ESX Server used for migration you get a message like 0x80000001 at edx.
Turn on or turn off VT on all ESX Servers used for VMotion and you´re fine.
I´ve got another information about migrating between
ESX Hosts.
If Intel VT is activated on only one of the ESX
Server used for migration you get a message like
0x80000001 at edx.
Turn on or turn off VT on all ESX Servers used for
VMotion and you´re fine.
Hi there,
You seem to know what you are doing - have thought of contributing to writing a whitepaper on this feature?
Regards
Mike
Something that you might be all interested in:
Richard at Anykey has written a VC SDK application which reports cpu compatiability information:
http://www.run-virtual.com/?p=153
Regards
Mike
The presentation is available now:
[url=http://download3.vmware.com/vmworld/2006/tac1356.
pdf]VMotion between Apples and Oranges, Understanding
CPU Compatibility Constraints for
VMware®VMotionTM[/url]
Lars
This was a great presentation - and I am glad its been released - I was hoping that I could inspire a community based whitepaper - which would take this PPT as a starting point - creating a cookbook for anyone want to use the "advanced" CPU masking features...
Regards
Mike
Hi Mike,
This is a good idea, I'd like to help however I can. With some coordinate with the autor of the vmotionInfo program on http://www.run-virtual.com we could get some auto-mask generation for VM migrations which would be very useful.
Obviously testing is limited by what kit I have to test things out on. I've been playing around a bit vmotioning VMs between 8832 HS20 IBM Blades and 8843 HS20's. Cpu info as per below.
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Xeon(TM) CPU 2.80GHz
stepping : 9
cpu MHz : 2799.302
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm
bogomips : 5583.66
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 2.80GHz
stepping : 3
cpu MHz : 2800.238
cache size : 2048 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm lm
bogomips : 5596.77
As you can see the newer processors have the lm bit. I've found that the following mask on the level 0x1 ecx bit allows vmotion between them.
Hi all,
I didn't have the opportunity to attend the session at VMworld 2006 but had already experimented with CPU masks before the conference and had written up some brief information here:
http://blog.scottlowe.org/2006/09/25/sneaking-around-vmotion-limitations/
After seeing the release of the VMotionInfo tool by Richard Garsthagen, I downloaded his tool and then expanded upon my original article to include more information. The expanded information is here:
http://blog.scottlowe.org/2006/11/23/vmotion-compatibility/
I'd love for the technical geniuses here to pick apart what I've written and contribute additional information based on their own experience. Perhaps this can form the basis of a community-written whitepaper? I'm willing to help in any way that I can.
Regards,
Scott
Thanks for the articles Scott, they helped out. Here's how I managed to get Vmotion working on my test system.
Trying to perform a Vmotion from a Dell 2950 to a Dell 2850 (fyi: according to the Dell chart here: http://download3.vmware.com/vmworld/2006/tac1356.pdf they are incompatible with VMotion).
Dell 2950 with 2 Intel(R) Xeon(R) CPU 5160 @ 3.00Ghz
Dell 2850 with 2 Intel(R) Xeon(TM) CPU 3.60Ghz
When attempting to perform the VMotion, I would see the following message:
The CPU of the host is incompatible with the CPU feature requirements of virtual machine; problem detected at CPUID level 0x1 register 'eax'.[/b]
First, I downloaded the VMotion Info utility (http://www.run-virtual.com/?page_id=155). The trick is to run it in debug mode (there is a option on the login page for this) so you can see the actual registers:
For the 2850 CPU 0 Level 1, eax register:
0000:0000:0000:0000:0000:1111:0100:0011
For the 2950 CPU 0 Level 1, eax register:
0000:0000:0000:0000:0000:0110:1111:0110
The different bits are:
0000:0000:0000:0000:0000:\*11*:*1**:0*1*
Then on the VM, Edit Settings...Options Tab...Advanced...Advanced Button. Click on Level 1 eax. The default mask for the EAX row (shown in the Row details section):
XXXX HHHH HHHH XXXX XXXX HHHH XXXX XXXX
X bits are not used by the VM, the H bits are shown to the VM, so basically we can ignore any different bits that have an X. To disable the feature, we want to set the conflicting bits to 0. The resulting bit mask is:
\-- -
-
XXXX HHHH HHHH XXXX XXXX 0HH0 XXXX XXXX
After saving the changes and attempting to VMotion again, I would see the following message:
The CPU of the host is incompatible with the CPU feature requirements of virtual machine; problem detected at CPUID level 0x1 register 'ecx'.[/b]
Checking the VMotion Info Utility for the ecx register differences:
For the 2850 CPU 0 Level 1, ecx register:
0000:0000:0000:0000:0110:0101:1001:1101
For the 2950 CPU 0 Level 1, ecx register:
0000:0000:0000:0100:1110:0011:1011:1101
The different bits are:
0000:0000:0000:0*00:*110:0**1:1011:1101
The default mask for the ECX register (again, click on the register and see the default in the Row details section below):
RRRR RRRR RRRR RRR0 00XR R0H0 000H 0RRH
R bits are already hidden from the OS, X bits are not used by the VM, and the 0 bits are already hidden, so we only need to mask out the conflicting H bits with a zero:
\-- -
--0- -
-
RRRR RRRR RRRR R0R0 00XR R000 000H 0RRH
After saving the changes and attempting to VMotion again, it works! Well it seems to work anyway. The trick now is to figure out if I'm worried that making VMotion ignore those bits is going to cause me problems later on. I'm probably guessing that it will cause me problems, so I won't do this on my production environment. Fun to do in testing though.
For the EAX register, I changed bits 8 & 11 (the processor family identifier bits).
For the ECX register, I changed bits:
9 - SSSE3 (Which is what VMotion Info found as incompatible)
18 - DCA (Direct Cache Access)
Forge,
Good information, thanks for sharing that with everyone. Programmers/developers out there: What could be the potential implications of hiding these bits (and thus masking the CPU's true functionality)? I'm thinking that in most cases it would simply cause a slight decrease in performance, but does anyone know if serious stability issues will result? That seems to be the #1 question these days, and one I'm not prepared to answer.
Regards,
Scott