ESXi inside KVM - This Virtual Machine is configur...

pointer_00 · ‎01-16-2014

Hi All,

I am using nested virtualization where I have installed ESXi 5.5 inside KVM. However,I am unable to spawn a VM with 64-bit VM (ubuntu 12.04) on my ESXi host and getting the following error:

"This Virtual Machine is configured for 64-bit operating systems. However, 64-bit operation is not possible. This host does not support Intel VT-x."

I have the following configs set in .vmx file of the VM:
- vhv.enable = "TRUE" and
- vmx.allowNested = "TRUE
I also have vhv.allow = "TRUE" set in /etc/vmware/config. (Though, its mentioned in some of your threads [https://communities.vmware.com/message/2220460#2220460] that this option is not needed in ESXi 5.1 and above.
Regarding my machine settings:
- Supports 64-bit Longmode : Yes
- Supports 64-bit VMware : Yes
- Supported EVC Modes : None

and;

$> cat /sys/module/kvm_intel/parameters/nested

Y

$ cat /sys/module/kvm_intel/parameters/ept

Y

The vmware log is as follows:

(Complete vmware.log attached)

2014-01-16T08:41:29.345Z| vmx| I120: Tools: sending 'OS_PowerOn' (state = 3) state change request

2014-01-16T08:41:29.345Z| vmx| I120: Tools: Delaying state change request to state 3.

2014-01-16T08:41:29.345Z| vmx| I120: TOOLS INSTALL initializing state to IDLE on power on.

2014-01-16T08:41:29.357Z| vmx| I120: FeatureCompat: VM masks:

2014-01-16T08:41:29.357Z| vmx| I120: MonPmc: ctrBase 0xc1 selBase 0x186/1 PGC 0/0 SMM 0 drain 0 flush 0

2014-01-16T08:41:29.357Z| vmx| I120+ MonPmc:   gen counters num: 2 width 0 write width 0

2014-01-16T08:41:29.357Z| vmx| I120+ MonPmc:   fix counters num: 0 width 0

2014-01-16T08:41:29.357Z| vmx| I120+ MonPmc:   unavailable counters: 0

2014-01-16T08:41:29.359Z| vmx| I120: guestCpuFeatures = 0xf8

2014-01-16T08:41:29.364Z| vmx| I120: Guest CPUID differences from hostCPUID.

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 00000001, 0: 0x00000663 0x00010800 0x80800001 0x078bfbfd

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 00000006, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 00000007, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 00000008, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 00000009, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000a, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000b, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000c, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 3: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 4: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 5: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 6: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 7: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 8: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 9: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, a: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, b: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, c: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, d: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, e: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, f: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 10: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 11: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 12: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 13: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 14: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 15: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 16: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 17: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 18: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 19: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 1a: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 1b: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 1c: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 1d: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 1e: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 1f: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 20: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 21: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 22: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 23: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 24: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 25: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 26: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 27: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 28: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 29: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2a: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2b: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2c: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2d: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2e: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 2f: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 30: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 31: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 32: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 33: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 34: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 35: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 36: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 37: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 38: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 39: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 3a: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 3b: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 3c: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 3d: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 0000000d, 3e: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000000, 0: 0x40000010 0x61774d56 0x4d566572 0x65726177

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000001, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000002, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000003, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000004, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000005, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000006, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000007, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000008, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000009, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 4000000a, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 4000000b, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 4000000c, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 4000000d, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 4000000e, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 4000000f, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 40000010, 0: 0x00260a9e 0x000101d0 0x00000000 0x00000000

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 80000001, 0: 0x00000663 0x00000000 0x00000000 0x0191abfd

2014-01-16T08:41:29.364Z| vmx| I120: guestCPUID level 80000007, 0: 0x00000000 0x00000000 0x00000000 0x00000100

2014-01-16T08:41:29.364Z| vmx| I120: Msg_Post: Error

2014-01-16T08:41:29.364Z| vmx| I120: [msg.cpuid.noLongmode2] This virtual machine is configured for 64-bit guest operating systems. However, 64-bit operation is not possible.

2014-01-16T08:41:29.364Z| vmx| I120+ This host does not support Intel VT-x.

2014-01-16T08:41:29.364Z| vmx| I120+ For more detailed information, see http://vmware.com/info?id=152.

2014-01-16T08:41:29.364Z| vmx| I120: ----------------------------------------

2014-01-16T08:41:29.370Z| vmx| I120: Module CPUID power on failed.

2014-01-16T08:41:29.370Z| vmx| I120: VMX_PowerOn: ModuleTable_PowerOn = 0

2014-01-16T08:41:29.376Z| vmx| I120: SVMotion_PowerOff: Not running Storage vMotion. Nothing to do

2014-01-16T08:41:29.376Z| svga| I120: SVGA thread is exiting

2014-01-16T08:41:29.376Z| vmx| I120: Destroying virtual dev for scsi0:0 vscsi=8196

2014-01-16T08:41:29.376Z| vmx| I120: VMMon_VSCSIStopVports: No such target on adapter

2014-01-16T08:41:29.377Z| mks| I120: MKS PowerOff

2014-01-16T08:41:29.377Z| mks| I120: MKS thread is exiting

2014-01-16T08:41:29.377Z| vmx| I120: scsi0:0: numIOs = 0 numMergedIOs = 0 numSplitIOs = 0 ( 0.0%)

2014-01-16T08:41:29.377Z| vmx| I120: Closing disk scsi0:0

2014-01-16T08:41:29.378Z| vmx| I120: DISKLIB-VMFS : "/vmfs/volumes/52d413a8-96276407-888d-5254002a4c29/vm1/vm1-flat.vmdk" : closed.

2014-01-16T08:41:29.378Z| vmx| I120: SVMotion_PowerOff: Not running Storage vMotion. Nothing to do

2014-01-16T08:41:29.381Z| vmx| I120: WORKER: asyncOps=2 maxActiveOps=1 maxPending=1 maxCompleted=1

2014-01-16T08:41:29.381Z| vmx| I120: Vix: [39054 mainDispatch.c:1201]: VMAutomationPowerOff: Powering off.

2014-01-16T08:41:29.382Z| vmx| W110: /vmfs/volumes/52d413a8-96276407-888d-5254002a4c29/vm1/vm1.vmx: Cannot remove symlink /var/run/vmware/root_0/1389861689012676_39054/configFile: No such file or directory

2014-01-16T08:41:29.405Z| vmx| I120: Vix: [39054 mainDispatch.c:3964]: VMAutomation_ReportPowerOpFinished: statevar=1, newAppState=1873, success=1 additionalError=0

2014-01-16T08:41:29.405Z| vmx| I120: Vix: [39054 mainDispatch.c:3964]: VMAutomation_ReportPowerOpFinished: statevar=0, newAppState=1870, success=1 additionalError=0

2014-01-16T08:41:29.405Z| vmx| I120: Transitioned vmx/execState/val to poweredOff

2014-01-16T08:41:29.405Z| vmx| I120: Vix: [39054 mainDispatch.c:3964]: VMAutomation_ReportPowerOpFinished: statevar=0, newAppState=1870, success=0 additionalError=0

2014-01-16T08:41:29.405Z| vmx| I120: Vix: [39054 mainDispatch.c:4003]: Error VIX_E_FAIL in VMAutomation_ReportPowerOpFinished(): Unknown error

2014-01-16T08:41:29.405Z| vmx| I120: Vix: [39054 mainDispatch.c:3964]: VMAutomation_ReportPowerOpFinished: statevar=0, newAppState=1870, success=1 additionalError=0

2014-01-16T08:41:29.405Z| vmx| I120: Transitioned vmx/execState/val to poweredOff

2014-01-16T08:41:29.405Z| vmx| I120: VMIOP: Exit

2014-01-16T08:41:29.433Z| vmx| I120: Vix: [39054 mainDispatch.c:849]: VMAutomation_LateShutdown()

2014-01-16T08:41:29.433Z| vmx| I120: Vix: [39054 mainDispatch.c:799]: VMAutomationCloseListenerSocket. Closing listener socket.

2014-01-16T08:41:29.437Z| vmx| I120: Flushing VMX VMDB connections

2014-01-16T08:41:29.437Z| vmx| I120: VmdbDbRemoveCnx: Removing Cnx from Db for '/db/connection/#1/'

2014-01-16T08:41:29.437Z| vmx| I120: VmdbCnxDisconnect: Disconnect: closed pipe for pub cnx '/db/connection/#1/' (0)

2014-01-16T08:41:29.446Z| vmx| I120: VMX exit (0).

2014-01-16T08:41:29.446Z| vmx| I120: AIOMGR-S : stat o=1 r=3 w=0 i=0 br=49152 bw=0

2014-01-16T08:41:29.446Z| vmx| I120: OBJLIB-LIB: ObjLib cleanup done.

2014-01-16T08:41:29.447Z| vmx| W110: VMX has left the building: 0.

Any help would be much appreciated..

admin · ‎01-16-2014

pointer_00 wrote:

Hi All,

I am using nested virtualization where I have installed ESXi 5.5 inside KVM. However,I am unable to spawn a VM with 64-bit VM (ubuntu 12.04) on my ESXi host and getting the following error:

"This Virtual Machine is configured for 64-bit operating systems. However, 64-bit operation is not possible. This host does not support Intel VT-x."

I have the following configs set in .vmx file of the VM:

vhv.enable = "TRUE" and

vmx.allowNested = "TRUE

I also have vhv.allow = "TRUE" set in /etc/vmware/config. (Though, its mentioned in some of your threads [https://communities.vmware.com/message/2220460#2220460] that this option is not needed in ESXi 5.1 and above.

The ESXi vhv settings are irrelevant on this scenario. They are used when running ESXi (or some other VMware product) on the physical hardware and you want to virtualize hardware-assisted virtualization for a guest hypervisor under ESXi.

In your situation, you need to configure *kvm* to virtualize hardware-assisted virtualization for the ESXi guest hypervisor running under kvm.

In /etc/modprobe.d, you need the following settings:

In kvm.conf:

options kvm ignore_msrs=1

In kvm-intel.conf:

options kvm-intel nested=y ept=y

pointer_00 · ‎01-16-2014

Thanks alot for you reply. I have all the flags enabled.

$> cat /sys/module/kvm_intel/parameters/nested

Y

$ cat /sys/module/kvm_intel/parameters/ept

Y

$> cat /sys/module/kvm/parameters/ignore_msrs

Y

However, running/spawning a VM inside KVM (using CPU identification utility iso: VMware vSphere 4: Private Cloud Computing, Server and Data Center Virtualization) shows:

Supports 64-bit Longmode : Yes
Supports 64-bit VMware : No
Supported EVC Modes : None

admin · ‎01-16-2014

From the log file you posted above, kvm is not virtualizing VT-x/EPT.

Bit 5 of CPUID(1).ecx is clear (indicating no VT-x support):

2014-01-16T08:41:29.044Z| vmx| I120: hostCPUID level 00000001, 0: 0x00000663 0x00000800 0x80802001 0x078bfbfd

And none of the VMX capability MSRs are present:

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x480 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x481 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x482 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x483 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x484 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x485 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x486 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x487 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x488 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x489 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x48a = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x48b = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x48c = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x48d = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x48e = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x48f = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x490 = 0

2014-01-16T08:41:29.044Z| vmx| I120: Common: MSR 0x491 = 0

I am not sure what additional kvm configuration is necessary to get this to work.

I see that kvm is emulating a Pentium3:

2014-01-16T08:41:29.043Z| vmx| I120: hostCPUID codename: Pentium 3 -- Unknown codename

2014-01-16T08:41:29.043Z| vmx| I120: hostCPUID name: QEMU Virtual CPU version 1.7.0

Of course, Pentium 3's did not have VT-x support. Maybe you need an option to tell it to emulate your physical processor?

pointer_00 · ‎01-16-2014

Ok, so there was an option in virt-manager to pick up the processor type for the guest OS. (By default it was picking up the pentium 3 one). I picked up core2duo.

After installing ESXi host with it (I haven't tried installing the VM on top of it yet), I am getting the following:

~ # esxcfg-info | grep "HV"

|----HV Support............................................2

|----HV Replay Capable.....................................false

\==+HV Replay Disabled Reasons :

|----Reason.............................................HV disabled in BIOS

|----Group Name....................CPUHV

|----World Command Line.................................grep HV

Does this mean again, that it will create trouble for the VMs?

admin · ‎01-16-2014

pointer_00 wrote:

         \==+HV Replay Disabled Reasons :

            |----Reason.............................................HV disabled in BIOS

The kvm virtual machine has marked VT-x disabled in the virtual feature control MSR. Perhaps there are setting in the virtual BIOS to enable this?

pointer_00 · ‎01-17-2014

I tried this with no luck. Couldn't find any option to enable it.

pointer_00 · ‎01-17-2014

Btw, when I tried to boot the VM with these settings now, first I got this in the vmware.log:

"You are running VMware ESX through an incompatible hypervisor. You cannot power on a virtual machine until this hypervisor is disabled."

I then added vmx.allowNested = "TRUE" to the .vmx file of the VM and the error is changed to (different from the one mentioned in the title of the thread):

"This Virtual Machine is configured for 64-bit operating systems. However, 64-bit operation is not possible. This host supports Intel VT-X, but the Intel VT-x implementation is incompatible with VMWare ESX."

Detailed vmware.log for the second case attached.

admin · ‎01-17-2014

Okay, now we're getting somewhere. ESXi is not satisfied with kvm's implementation of VT-x. Before I dig through the hexadecimal output in the log file to figure out which features are missing, try adding the following configuration option to the nested VM:

hv.assumeEnabled = TRUE

I believe that should get us some more human-readable output regarding the VT-x capabilities of your virtual CPU.

Please post the resulting vmware.log file.

pointer_00 · ‎01-17-2014

try adding the following configuration option to the nested VM:

hv.assumeEnabled = TRUE

Did this. Seeing the error:

"[msg.cpuid.noLongmode2] This virtual machine is configured for 64-bit guest operating systems. However, 64-bit operation is not possible.

2014-01-17T17:10:18.670Z| vmx| I120+ This host supports Intel VT-x, but the Intel VT-x implementation is incompatible with VMware ESX."

vmware.log attached.

admin · ‎01-17-2014

Well, unfortunately, that did not get the human-readable dump of your virtual processor's capabilities that I was hoping for.

Decoding the MSR values from the log, I've found the following reasons that ESXi is dissatisfied with the VT-x implementation of the kvm virtual CPU:

1. We require several Primary Processor-Based VM-Execution Controls not implemented by the kvm virtual processor. These include:

Bits 1, 4, 5, 6, 8, 13, 14, 26. <Intel NDA.>

Bit 11: RDPMC exiting.

Bit 12: RDTSC exiting.

Bit 21: Use TPR shadowing.

2. We require several VM-Exit Controls not implemented by the kvm virtual processor. These include:

Bits 0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17. <Intel NDA.>

Bit 15: Acknowledge interrupt on exit.

3. We require several VM-Entry Controls not implemented by the kvm virtual processor. These include:

Bits 0, 1, 3, 4, 5, 6, 7, 8, 12. <Intel NDA.>

Until kvm supports these features, you will not be able to run nested 64-bit VM's under ESXi under kvm on Intel hardware.

pointer_00 · ‎01-19-2014

Too bad, I'll try out the other options (other than KVM) instead..

but thanks a lot for all your help

admin · ‎05-22-2014

pointer_00 wrote:

try adding the following configuration option to the nested VM:

hv.assumeEnabled = TRUE

Did this. Seeing the error:

"[msg.cpuid.noLongmode2] This virtual machine is configured for 64-bit guest operating systems. However, 64-bit operation is not possible.

2014-01-17T17:10:18.670Z| vmx| I120+ This host supports Intel VT-x, but the Intel VT-x implementation is incompatible with VMware ESX."

vmware.log attached.

Oops. That option is only respected if it's in the /etc/vmware/config file.

With hv.assumeEnabled=TRUE in /etc/vmware/config, the vmware.log of the guest should enumerate the missing VT-x features of the virtual CPU.

AsmPro · ‎01-28-2016

Hi jmattson,

I am trying to do the same as previous guys, that is install ESXi 6.0U1 as nested HV under KVM on Ubuntu 14.04LTS, which is using kernel 3.13. I have also patched qemu with vmport=off option, so CPUID would not be emulated by the kvm, but queried directly.

I tried to expose both host cpu (pass-through) and core2duo with 1 socket, 2 cores and 2 threads. Original CPU is Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz.

Content of /etc/vmware/config:

libdir = "/usr/lib/vmware"

authd.proxy.nfc = "vmware-hostd:ha-nfc"

authd.proxy.nfcssl = "vmware-hostd:ha-nfcssl"

authd.proxy.vpxa-nfcssl = "vmware-vpxa:vpxa-nfcssl"

authd.proxy.vpxa-nfc = "vmware-vpxa:vpxa-nfc"

authd.fullpath = "/sbin/authd"

hv.assumeEnabled = "TRUE"

vhv.allow = "TRUE"

VMX of the VM contions the following additional config option:

vmx.allowNested = "TRUE"

I attached the vmware.log of the VM reporting error (win10 64-bit).

Thank you in advance for all your help.

Regards,

Uros

admin · ‎01-28-2016

AsmPro -

The vmware.log file shows:

2016-01-28T13:16:15.393Z| vmx| I120: The following VT-x features are required for VT-x support in VMware ESX; however, these features are not available on this host:

2016-01-28T13:16:15.393Z| vmx| I120: Primary Processor-Based VM-Execution Controls:

2016-01-28T13:16:15.393Z| vmx| I120: <bit 1>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 4>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 5>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 6>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 8>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 13>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 14>

2016-01-28T13:16:15.393Z| vmx| I120: <bit 26>

2016-01-28T13:16:15.393Z| vmx| I120: VM-Exit Controls:

2016-01-28T13:16:15.393Z| vmx| I120: Acknowledge interrupt on exit

I believe this is all fixed in a later version of kvm/qemu. I have been using QEMU emulator version 2.3.0 with Linux kernel 4.2.3, and it seems to work fine.

AsmPro · ‎01-28-2016

Thanx for the info. Will try that.

Regards,

Uros

AsmPro · ‎01-28-2016

I confirm it. It works with at least kernel 4.3, which cat be installed from DEB packages using instructions from page:

Linux Kernel 4.3 Released, How to Install / Upgrade it in Ubuntu | UbuntuHandbook

and with the original qemu-kvm and libvirt-bin packages which are shipped with Ubuntu 14.04LTS.

Well I did and used in a VM vmport=off patch in qemu-kvm, but not sure if this helped at all.

Regards,

Uros

Onryo · ‎03-01-2016

I am successfully running kvm on GNU/Linux Debian. I am using vfio to pass a GPU (Nvidia GTX970) bare metal to a Windows 10 machine. On the kvm Windows 10 vm I have installed vmware workstation 12.

I see you are using virt-manager on your GNU/Linux host. In the drop down to pick a CPU you will not find what you need to manually enter in the CPU field.

host-passthrough

This will expose your real CPU. The reason it is not in the virt-manager drop down is because GNU/Linux encourages high availability of virtual machines. This is useful for applications such as a SAN when another computer using some other real hardware needs to use your image in a split second.

You can see in my vmware logs that my cpu is correctly seen though kvm.

[code]

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID vendor: GenuineIntel

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID family: 0x6 model: 0x3f stepping: 0x2

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID codename: Haswell EP/EN/EX

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID name: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID level 00000000, 0: 0x0000000d 0x756e6547 0x6c65746e 0x49656e69

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID level 00000001, 0: 0x000306f2 0x00080800 0xfffa3223 0x1f8bfbff

2016-02-29T18:42:15.919+01:00| vmx| I125: hostCPUID level 00000002, 0: 0x76036301 0x00f0b5ff 0x00000000 0x00c10000

[/code]

For whatever reasons nesting vmwares products on kvm is not enabled by default. kvm (kernel virtual machine) is standard stuff so I find this odd. You need to add.

vmx.allowNested = “TRUE”

And as you correctly stated above make sure your GNU/Linux host is offering to nest.

$> cat /sys/module/kvm_intel/parameters/nested

Y

$ cat /sys/module/kvm_intel/parameters/ept

Y

You might find in your dmesg on the GNU/Linux host some programs in the VM or nested VM will probe and poke around in stuff that is not there. To help avoid problems ignore their noise.

options kvm ignore_msrs=1

admin · ‎03-01-2016

For whatever reasons nesting vmwares products on kvm is not enabled by default. kvm (kernel virtual machine) is standard stuff so I find this odd.

To date, VMware products running on a kvm virtual CPU do not pass our quality testing. However, we recognize that correctness is not of primary importance to people running in this configuration, so you can expect this restriction to eventually be removed.

it_dir · ‎11-13-2021

@admin wrote:
... you can expect this restriction to eventually be removed.

Any idea when?

All

ESXi inside KVM - This Virtual Machine is configured for 64-bit operating systems. However, 64-bit operation is not possible. This host does not support Intel VT-x.