1 2 3 4 5 Previous Next 180 Replies Latest reply on Sep 26, 2016 6:08 AM by Hejdesgaard Go to original post
      • 30. Re: ESXi inside KVM
        zamf Novice


        It turns out there is an option to ignore the bus speed difference: One can pass busSpeedMayVary=TRUE in the boot options. I installed ESXi with the QEMU options --enable-kvm -smp 2 and then booted the image without KVM with the options: "-cpu SandyBridge -smp 2" and passed the option busSpeedMayVary=TRUE to ESXi at boot time.

        This solved the problem.


        I am still searching for a way to boot ESXi 5.5 inside a QEMU vm with only 1 CPU. Here is the message I get when I try to run with a single CPU.

        Screenshot 2013-11-14 14.46.10.png


        Thank you!

        • 31. Re: ESXi inside KVM

          This message is from the installer.  What if you install under KVM with two vCPUs and then try to boot the already-installed image under QEMU with a single vCPU?

          • 32. Re: ESXi inside KVM
            zamf Novice

            Thanks  a lot, this worked. Only the installer checks the number of CPUs.

            • 33. Re: ESXi inside KVM
              zamf Novice

              ESXi boots just fine in both KVM and dynamic binary translation mode with versions of QEMU up to 1.0.

              However, we have a modified version of QEMU, and with this version, we get this ESXi crash.


              The serial log (attached) has various warnings about failed reads, like "Failed to request module '/usr/lib/vmware/vmkmod/iodm': Failure "


              Do you have any idea if this is the likely cause of the crash? Do you see any other suspicious behavior in the log?




              Thank you,


              • 34. Re: ESXi inside KVM

                Would you mind uploading the log file as an attachment rather than embedded in your message?  (Use the advanced editor.)

                • 35. Re: ESXi inside KVM

                  I suspect it means that networking did not come up.  What kind of virtual NIC are you using?

                  • 36. Re: ESXi inside KVM
                    zamf Novice

                    It is Intel e1000, which works with the vanilla QEMU.

                    Which message in the serial log triggered your suspicion? The serial log for a successful run with vanilla QEMU does not have messages like "Failed to request module '/usr/lib/vmware/vmkmod/iodm", so I was suspecting there is a problem with the disk.

                    • 37. Re: ESXi inside KVM

                      I'm looking at "Net: 1999: invalid Net_Create: class etherswitch not supported."  Since the backtrace shows a page fault in NetVsi_TcpipMultiInstanceList, I'm guessing that the error is networking related.

                      • 38. Re: ESXi inside KVM
                        zamf Novice

                        I did a diff of the serial log for a successful run and a failed run. Besides the fact that the failed run does not detect the HPET, this is the first divergence:


                        Correct run:

                        cpu0:33234)Loading module iodm ...

                        cpu0:33234)Elf: 1861: module iodm has license VMware

                        cpu0:33234)Mod: 4780: Initialization of iodm succeeded with module ID 4.

                        cpu0:33234)iodm loaded successfully.

                        Failed run:

                        cpu0:33232)User: 2886: wantCoreDump : vmkeventd -enabled : 0

                        cpu0:33154)WARNING: Mod: 6592: Failed to request module '/usr/lib/vmware/vmkmod/iodm': Failure

                        cpu0:33154)Config: 346: "UserMemASRandomSeed" = 1965899852, Old Value: 0, (Status: 0x0)


                        This error occurs just after the vmkapi_mgmt module loaded successfully in both runs. Strangely, the iodm module eventually gets to be loaded in the failed run too, after all sort of other failures to request other modules.

                        cpu0:33283)Loading module vmkplexer ...

                        cpu0:33283)Elf: 1861: module vmkplexer has license GPLv2

                        cpu0:33283)vmkplexer-heap: initial size : 262144, max size: 20971520

                        cpu0:33283)vmkplexer-heap: heap creation succeeded. id = 0x41090eebe000

                        cpu0:33283)vmkplexer registration succeeded!

                        cpu0:33283)Mod: 4780: Initialization of vmkplexer succeeded with module ID 4101.

                        cpu0:33283)vmkplexer loaded successfully.

                        cpu0:33283)Loading module iodm ...

                        cpu0:33283)Elf: 1861: module iodm has license VMware

                        cpu0:33283)Mod: 4780: Initialization of iodm succeeded with module ID 6.

                        cpu0:33283)iodm loaded successfully.


                        However, many modules are not loaded, including networking related ones. The error you pointed out probably comes from the fact that this module could not be loaded:

                        Failed to request module '/usr/lib/vmware/vmkmod/etherswitch'


                        I think the root cause is related to the failed module requests from /usr/lib

                        My question is where are these modules located? Is it a ram disk or the disk where I installed ESXi?

                        Can I enable additional logging related to these failed module requests?


                        Thank you,


                        • 39. Re: ESXi inside KVM

                          It would appear that vmkeventd is dying, and the messages about modules failing to load are the result of not being able to communicate with vmkeventd.  I don't really see any way of enabling additional logging, aside from booting a beta build of ESXi.

                          • 40. Re: ESXi inside KVM
                            zamf Novice

                            Thanks a lot for your help.  Should I consider this an ESXi bug and report it?  We have been able to run many flavors of operating systems in our version of QEMU, so it is strange ESXi crashes. Perhaps this could be detected by an assert instead of generating a PSOD. Moreover, perhaps by generating a coredump and reporting the bug, I could get more insights into the root cause of the problem. What do you think?

                            Please let me know what is the best way to report the probelm. Should I use the FTP instructions here? http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1008525#AddEmail


                            • 41. Re: ESXi inside KVM

                              I don't think there's enough evidence to ascertain whether the issue is in ESXi or in the emulator. A beta build of ESXi may have an assertion to cover this, but if there is an emulation bug, it's unlikely.  We typically don't add assertions to verify hardware behavior.


                              If you can't reproduce the problem on supported hardware, you probably won't get much traction with a bug report.  Send me the information on how to reproduce the problem, and I'll try to take a look when I get some time.

                              • 42. Re: ESXi inside KVM
                                brendanpg Lurker


                                I got everything but networking working with 5.5rollup1 so far on KVM (kernel 3.12.6, libvirt 1.2.0, QEMU 1.7.0, plus the Qemu piix patch from above), plus the following kvm flags:

                                modprobe.d]# cat kvm.conf

                                options kvm ignore_msrs=1

                                modprobe.d]# cat kvm-intel.conf

                                options kvm-intel nested=y ept=y


                                The last piece I am stuck on is the networking.


                                I tried Realtek, e1000 and vmxnet on the kvm side.


                                None of these work with ESXi 5.5, and I cant figure out why. I see some packets, ping will work in some cases - I see truncated packets and larger packets simply dont work. Any ideas on what I can do to get any basic networking - the test harness doesnt need super-high performance.

                                • 43. Re: ESXi inside KVM
                                  mathiasew Lurker

                                  Did anyone manage to get it to work? I am desperate for a solution

                                  • 44. Re: ESXi inside KVM
                                    mathiase Enthusiast

                                    nothing yet? who can I bribe to make to work?

                                    1 2 3 4 5 Previous Next