It turns out there is an option to ignore the bus speed difference: One can pass busSpeedMayVary=TRUE in the boot options. I installed ESXi with the QEMU options --enable-kvm -smp 2 and then booted the image without KVM with the options: "-cpu SandyBridge -smp 2" and passed the option busSpeedMayVary=TRUE to ESXi at boot time.
This solved the problem.
I am still searching for a way to boot ESXi 5.5 inside a QEMU vm with only 1 CPU. Here is the message I get when I try to run with a single CPU.
This message is from the installer. What if you install under KVM with two vCPUs and then try to boot the already-installed image under QEMU with a single vCPU?
Thanks a lot, this worked. Only the installer checks the number of CPUs.
ESXi boots just fine in both KVM and dynamic binary translation mode with versions of QEMU up to 1.0.
However, we have a modified version of QEMU, and with this version, we get this ESXi crash.
The serial log (attached) has various warnings about failed reads, like "Failed to request module '/usr/lib/vmware/vmkmod/iodm': Failure "
Do you have any idea if this is the likely cause of the crash? Do you see any other suspicious behavior in the log?
serial_s2e_esxi_crash.txt 217.7 K
Would you mind uploading the log file as an attachment rather than embedded in your message? (Use the advanced editor.)
I suspect it means that networking did not come up. What kind of virtual NIC are you using?
It is Intel e1000, which works with the vanilla QEMU.
Which message in the serial log triggered your suspicion? The serial log for a successful run with vanilla QEMU does not have messages like "Failed to request module '/usr/lib/vmware/vmkmod/iodm", so I was suspecting there is a problem with the disk.
I'm looking at "Net: 1999: invalid Net_Create: class etherswitch not supported." Since the backtrace shows a page fault in NetVsi_TcpipMultiInstanceList, I'm guessing that the error is networking related.
I did a diff of the serial log for a successful run and a failed run. Besides the fact that the failed run does not detect the HPET, this is the first divergence:
cpu0:33234)Loading module iodm ...
cpu0:33234)Elf: 1861: module iodm has license VMware
cpu0:33234)Mod: 4780: Initialization of iodm succeeded with module ID 4.
cpu0:33234)iodm loaded successfully.
cpu0:33232)User: 2886: wantCoreDump : vmkeventd -enabled : 0
cpu0:33154)WARNING: Mod: 6592: Failed to request module '/usr/lib/vmware/vmkmod/iodm': Failure
cpu0:33154)Config: 346: "UserMemASRandomSeed" = 1965899852, Old Value: 0, (Status: 0x0)
This error occurs just after the vmkapi_mgmt module loaded successfully in both runs. Strangely, the iodm module eventually gets to be loaded in the failed run too, after all sort of other failures to request other modules.
cpu0:33283)Loading module vmkplexer ...
cpu0:33283)Elf: 1861: module vmkplexer has license GPLv2
cpu0:33283)vmkplexer-heap: initial size : 262144, max size: 20971520
cpu0:33283)vmkplexer-heap: heap creation succeeded. id = 0x41090eebe000
cpu0:33283)vmkplexer registration succeeded!
cpu0:33283)Mod: 4780: Initialization of vmkplexer succeeded with module ID 4101.
cpu0:33283)vmkplexer loaded successfully.
cpu0:33283)Loading module iodm ...
cpu0:33283)Elf: 1861: module iodm has license VMware
cpu0:33283)Mod: 4780: Initialization of iodm succeeded with module ID 6.
cpu0:33283)iodm loaded successfully.
However, many modules are not loaded, including networking related ones. The error you pointed out probably comes from the fact that this module could not be loaded:
Failed to request module '/usr/lib/vmware/vmkmod/etherswitch'
I think the root cause is related to the failed module requests from /usr/lib
My question is where are these modules located? Is it a ram disk or the disk where I installed ESXi?
Can I enable additional logging related to these failed module requests?
It would appear that vmkeventd is dying, and the messages about modules failing to load are the result of not being able to communicate with vmkeventd. I don't really see any way of enabling additional logging, aside from booting a beta build of ESXi.
Thanks a lot for your help. Should I consider this an ESXi bug and report it? We have been able to run many flavors of operating systems in our version of QEMU, so it is strange ESXi crashes. Perhaps this could be detected by an assert instead of generating a PSOD. Moreover, perhaps by generating a coredump and reporting the bug, I could get more insights into the root cause of the problem. What do you think?
Please let me know what is the best way to report the probelm. Should I use the FTP instructions here? http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1008525#AddEmail
I don't think there's enough evidence to ascertain whether the issue is in ESXi or in the emulator. A beta build of ESXi may have an assertion to cover this, but if there is an emulation bug, it's unlikely. We typically don't add assertions to verify hardware behavior.
If you can't reproduce the problem on supported hardware, you probably won't get much traction with a bug report. Send me the information on how to reproduce the problem, and I'll try to take a look when I get some time.
I got everything but networking working with 5.5rollup1 so far on KVM (kernel 3.12.6, libvirt 1.2.0, QEMU 1.7.0, plus the Qemu piix patch from above), plus the following kvm flags:
modprobe.d]# cat kvm.conf
options kvm ignore_msrs=1
modprobe.d]# cat kvm-intel.conf
options kvm-intel nested=y ept=y
The last piece I am stuck on is the networking.
I tried Realtek, e1000 and vmxnet on the kvm side.
None of these work with ESXi 5.5, and I cant figure out why. I see some packets, ping will work in some cases - I see truncated packets and larger packets simply dont work. Any ideas on what I can do to get any basic networking - the test harness doesnt need super-high performance.
Did anyone manage to get it to work? I am desperate for a solution
nothing yet? who can I bribe to make to work?