Thanks a lot for your help. Should I consider this an ESXi bug and report it? We have been able to run many flavors of operating systems in our version of QEMU, so it is strange ESXi crashes. Perhaps this could be detected by an assert instead of generating a PSOD. Moreover, perhaps by generating a coredump and reporting the bug, I could get more insights into the root cause of the problem. What do you think?
Please let me know what is the best way to report the probelm. Should I use the FTP instructions here? http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&e...
Thanks!
I don't think there's enough evidence to ascertain whether the issue is in ESXi or in the emulator. A beta build of ESXi may have an assertion to cover this, but if there is an emulation bug, it's unlikely. We typically don't add assertions to verify hardware behavior.
If you can't reproduce the problem on supported hardware, you probably won't get much traction with a bug report. Send me the information on how to reproduce the problem, and I'll try to take a look when I get some time.
Hi,
I got everything but networking working with 5.5rollup1 so far on KVM (kernel 3.12.6, libvirt 1.2.0, QEMU 1.7.0, plus the Qemu piix patch from above), plus the following kvm flags:
modprobe.d]# cat kvm.conf
options kvm ignore_msrs=1
modprobe.d]# cat kvm-intel.conf
options kvm-intel nested=y ept=y
The last piece I am stuck on is the networking.
I tried Realtek, e1000 and vmxnet on the kvm side.
None of these work with ESXi 5.5, and I cant figure out why. I see some packets, ping will work in some cases - I see truncated packets and larger packets simply dont work. Any ideas on what I can do to get any basic networking - the test harness doesnt need super-high performance.
Did anyone manage to get it to work? I am desperate for a solution
nothing yet? who can I bribe to make to work? :smileygrin:
brendanpg wrote:
None of these work with ESXi 5.5, and I cant figure out why. I see some packets, ping will work in some cases - I see truncated packets and larger packets simply dont work. Any ideas on what I can do to get any basic networking - the test harness doesnt need super-high performance.
Can you provide more details on the missing packets? Let's start with ping. If you put a sniffer on the virtual network, what sort of traffic do you see?
They were truncated packets. I have to revive the setup for getting the actual packet data. What would happen would be a few successful pings (sometimes it would ping, sometimes it would not) and then the packets would be truncated and anything larger than a minimum sized packet would be truncated. .
With ping it would say on the ESXi side "20 bytes missing" or something of this nature when dumping vmk.
Did the truncated packets come from the ESXi guest or from the kvm virtual switch or both?
Hi, I am just trying to catch up with the so far made progress in this matter. I downloaded QEMU source, edited hw/i386/pc_piix.c according to your earlier post, compiled and booted from ISO with
qemu-img create -f raw esxi00.img 10G
qemu-system-x86_64 -cpu SandyBridge -smp 2 -boot dc -m 2048 -hda esxi00.img -cdrom VMware-VMvisor-Installer-5.5.0-1331820.x86_64.iso
Now it is booting from the ISO and half way through the boot process making constant progress. As of writing this the screen shows "vmkibft loaded successfully". But to get there it took around 30min :smileygrin: any idea why it is that slow?
I would like to get to the point where the frames are truncated and then work from there.
cheers
Mathias
mathiase wrote:
Now it is booting from the ISO and half way through the boot process making constant progress. As of writing this the screen shows "vmkibft loaded successfully". But to get there it took around 30min :smileygrin: any idea why it is that slow?
That sounds like a good question for the KVM developers.
Yeah right :smileygrin: I was just wondering maybe you spit something out like "set flag XYZ starting qemu and you'll be good" So I am going to wait for it and then never shut my machine down again :smileygrin: You will probably hear from me when the installer completed. So long!
Hi, this is really strange: during installation I type 1234567 as the password and after installation the password doesnt work ... Can hardly be a layout issue, can it?
That is strange. I have no explanation.
Hi jmattson ;
I ran into the same problem on a physical Realtek with an injected VIB for ESXi 5.5.U1. Basically link is up, ethtool works on vmnic0 and all offloads are disabled, tcpdump shows truncated IP, and DHCP addresses dont work and static addresses dont work.
I get an error 49 cant unset IP, and then I get a no-router warning from ipv6.
So whats interesting is the virtual e1000 from qemu, the realtek from qemu and the 8168 realtek in physical are having the same issues - truncated IP, no DHCP, etc.
I've just attempted this too, and have had much the same experience... I am able to install and boot ESXi 5.5U1 inside KVM (qemu-1.7.0) after:
- Applying the patch to hw/i386/pc_piix.c from page 1 of this thread, and recompiling qemu
- Running "echo 1 > /sys/module/kvm/parameters/ignore_msrs" on the KVM host
...and have now hit the issue whereby networking doesn't seem to work (I'm presenting an e1000 nic to ESXi).
Although I did initially see "truncated-ip 246 bytes missing!" in the output from tcpdump-uw when run within ESXi, it looks like that is misleading and not the real problem. Running "tcpdump-uw -s0" so that it captures the entirety of the packets makes the "truncated-ip" messages go away.
What actually seems to be happening is that ESXi does not believe it is receiving any traffic from the network at all. It is successfully sending DHCP discovers (I have it configured to use DHCP), and they reach my DHCP server, which responds with DHCP offers. A tcpdump on the KVM host (on the virtual interface to which the ESXi guest is attached) shows that it is sending those DHCP offers towards the ESXi guest, but ESXi never sees them. Indeed the packet capture from within ESXi implies that it is not receiving any traffic whatsoever, including ARPs etc.
FWIW I'm also seeing the following show up regularly in /var/log/messages on the KVM host, at a rate of 1 per vCPU per second:
Apr 18 23:56:31 audi kernel: kvm [21198]: vcpu1 ignored rdmsr: 0x34
Apr 18 23:56:32 audi kernel: kvm [21198]: vcpu0 ignored rdmsr: 0x34
Actually, running "ethtool -S vmnic0" inside ESXi tells me that it is receiving packets. Just tcpdump-uw (ie capturing on vmk0) does see any of them arrive.
Edit: Although I'm not quite sure whether I believe ethtool. The first 5 lines of output actually say:
NIC statistics:
rx_packets: 755
tx_packets: 121
rx_bytes: 0
tx_bytes: 0
And the figures aren't incrementing despite continued traffic.
Message was edited by: alexmoore83
alexmoore83 wrote:
FWIW I'm also seeing the following show up regularly in /var/log/messages on the KVM host, at a rate of 1 per vCPU per second:
Apr 18 23:56:31 audi kernel: kvm [21198]: vcpu1 ignored rdmsr: 0x34
Apr 18 23:56:32 audi kernel: kvm [21198]: vcpu0 ignored rdmsr: 0x34
That's a query of the SMI count, which is probably just used for reporting purposes.
alexmoore83 wrote:
Actually, running "ethtool -S vmnic0" inside ESXi tells me that it is receiving packets. Just tcpdump-uw (ie capturing on vmk0) does see any of them arrive.
Edit: Although I'm not quite sure whether I believe ethtool. The first 5 lines of output actually say:
NIC statistics:
rx_packets: 755
tx_packets: 121
rx_bytes: 0
tx_bytes: 0
And the figures aren't incrementing despite continued traffic.
Message was edited by: alexmoore83
What MAC address is reported by esxcfg-vmknic -l? Is this the same as the MAC address that KVM has assigned to the ESXi VM?
jmattson wrote:
What MAC address is reported by esxcfg-vmknic -l? Is this the same as the MAC address that KVM has assigned to the ESXi VM?
Yes that reports the same MAC address as the one assigned by KVM (52:54:00:e1:fd:40 in both cases, which I believe was auto-generated by libvirt when I first defined the VM).
I don't know if this will be helpful, but can you post the output of
tcpdump -e ether src 52:54:00:e1:fd:40 or ether dst 52:54:00:e1:fd:40 (on the host)
while ESXi is negotiating for its DHCP address?