ESXi inside KVM

[LesykM]

Hello!

I wonder, if it is possible to run ESXi 5.0/5.1 inside KVM (proxmox) ?

I need to run few for testing purposes. I cannot setup any other virtualization software.

I need only ESXi itself, even without virtual machines inside it:)

Thanks!

[admin]

junkoi wrote:

i am wondering if there is any other KVM's NIC supported by ESXi (besides VMXnet3 & e1000), so i can try to see if the problem is gone?

thanks

I believe those are the only two choices.

[junkoi]

oh you really mean different "KVM host" (such as Fedora Linux), not "guest"?

i am not sure why this solves the problem, as others reported running exactly the same Ubuntu 14.04 for KVM host, and have no issue.

thanks.

[admin]

junkoi wrote:

oh you really mean different "KVM host" (such as Fedora Linux), not "guest"?

i am not sure why this solves the problem, as others reported running exactly the same Ubuntu 14.04 for KVM host, and have no issue.

thanks.

No; I was thinking of a Linux guest rather than ESXi, just to make sure that the network bridge was working properly.

[junkoi]

indeed, i have some doubts on my bridge setup.

- currently my host has IP 10.0.3.100

- i want to access to the guest (ESXi inside KVM) from the same network (10.0.3.xxx)

- i setup the bridge to get DHCP like below:

### /etc/network/interfaces - Ubuntu 14.04 - 64bit

auto lo

iface lo inet loopback

auto eth0

iface eth0 inet manual

auto br0

iface br0 inet dhcp

bridge_ports eth0

bridge_stp off

bridge_fd 0

bridge_maxwait 0

###

as a result, ESXi gets IP 10.0.3.101, and now i can SSH to it from 10.0.3.200

is there any wrong with this setup??

thanks.

[admin]

I think you should comment out these two lines:

# auto eth0

# iface eth0 inet manual

[junkoi]

oh, this is making some progress: with this new bridge setup (removing 2 lines as you suggested), i can go pass that "loading inventory" stage, and see the vSphere Client interface.

however, now if i try to create a new VM, the cursor keeps scrolling at the screen "Welcome to the new Virtual Machine wizard", and a message occur "Error: the command has timed out as the remote server is taking too long to respond".

perhaps there is some logs inside ESXi that i can look into to see what is the problem? is there anything like that?

thanks,

[junkoi]

so i tried again and again, and can confirm that the "Loading inventory" timed out issue is still there, as rarely i can go pass this to see the vSphere client screen.

thanks.

[junkoi]

come back to pure-Qemu emulator: i can run ESXi (installed with KVM) inside pure QEmu simply by removing the option "-enable-kvm".

however, there is a serious problem: at the console, the same password i can use with KVM-based ESXi is denied: the error message is "Invalid password".

this is very weird. any idea why this happens??

thanks

[admin]

No ideas.  Sorry.

[CloudMatt]

Hi again,

played a bit more and reproduced the setup on a Debian 7 system as the KVM host ... and funny things happened.

The ESX installation on the Debian 7 KVM host (with a patched qemu-2.0.0) went absolutely fine and ESX booted fine.

Very strange in this setup is that the Host cannot see/ping the ESX VM and visa versa BUT other systems on the same network can access/ping it (using the vmxnet3 network type). So there seems to be some issues with the networking but at least for me it is working fine on Ubuntu 14.04 (as KVM host).

thx and all the best,

Matt

[ahthrift]

Hi Matt,

We have ESX 5.5 installed on patched QEMU-2.0.0 running on Ubuntu 14.04 with bridged networking using the VMXNET3 driver. 

We are experiencing the same problems as you mention above:

- Cannot ping ESX from KVM Host

- Unreliable TCP communications to ESX (cannot load inventory, SSH pauses)

Its odd that we experience these problems on Ubuntu, while you experience them on Debian.   Maybe there is some minor configuration variance that causes this?

[mathiase]

I just installed QEMU 2.1 on Ubuntu Trusty. I got as far as a working DHCP from ESXi but from that point on no further traffic is possible. Problem seems to be that any ARP response does not make it into the ESXI arp cache. The cache is always empty while that of the KVM host trying to ping the ESXi virtual machine has the correct MAC installed.

tcpdump-uw on ESXi shows

IP truncated-ip - 2 bytes missing! 192.168.1.122.1 > 192.168.122.163: ICMP echo request ...

ARP, Request who has 192.168.122.1 tell 192.168.122.13, length 28

repeatedly while pinging ESXi from the KVM server.

esxcfg-nics confirmes ESXi has a vmxnet3 as vmnic0 at 1000Mbps Full, MTU 1500 and the MAC as configured by libvirt.

Did we find a solution for that? I skipped through the pages but couldn't find any.

cheers

Mathias

[admin]

I have found that my ESXi guest can connect to any system on the network except its host.  I haven't figured out a solution for this.

[CloudMatt]

Right, that is exactly what I found out too

from my previous post:

> Very strange in this setup is that the Host cannot see/ping the ESX VM and visa versa BUT other systems on the same network can access/ping it (using the vmxnet3 network type).

Haven't found a solution for this yet either

[mathiase]

So it does work for you to ping other VMs on the same host? If yes: Can you please check with Openvswitch instead of linux bridge?

How is it possible I see this from my virtual ESXi on my router

18:42:36.740661 52:54:00:af:de:3c > 52:54:00:22:07:f5, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 4686, offset 0, flags [none], proto ICMP (1), length 84)

    88.198.244.103 > 8.8.8.8: ICMP echo request, id 44185, seq 0, length 64

18:42:37.743222 52:54:00:af:de:3c > 52:54:00:22:07:f5, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 4687, offset 0, flags [none], proto ICMP (1), length 84)

    88.198.244.103 > 8.8.8.8: ICMP echo request, id 44185, seq 1, length 64

18:42:38.746666 52:54:00:af:de:3c > 52:54:00:22:07:f5, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 4688, offset 0, flags [none], proto ICMP (1), length 84)

    88.198.244.103 > 8.8.8.8: ICMP echo request, id 44185, seq 2, length 64

a proper ARP enty on the virtual ESXi but no ARP entry for the ESXi's IP address on the router?

root@r1:~# arp | grep 103

88.198.244.103                   (incomplete)                              eth2

root@r1:~#

cheers

Mathias

[mathiase]

Here is something else: It works with ESXi 6.0 Beta the E1000 chipset on a linux bridge (virbr0) and Open vSwitch. Had no luck with vmxnet3 though.

[gneville]

I too have been trying to get ESXi working inside KVM but haven't had much success until now.

I was getting the timeouts and disconnects when connecting to the ESXi host from vSphere Client, but I've just installed ESXi 6 beta and now it works okay.

However I'm having trouble getting 64-bit guests to power on. I keep getting errors such as "longmode is unsupported" or the implementation of vt-x is unsupported.

I've added these 4 switches to my /etc/vmware/config file but no change:

vmx.allowNested = TRUE

vhv.alow = TRUE

vhv.enable = TRUE

hv.assumeEnabled = TRUE

I know that my Host (running qemu) supports vt-x and EPT, I see vmx when in /proc/cpuinfo and the "Y" flag in /sys/module/kvm_intel/parameters/ept.

However, I see "HV Support ..... 1" when running esxcfg-info|grep "HV Support" inside the ESX guest meaning it might be available but not supported.

Any ideas please?

Many Thanks

[mathiase]

Does you virtual ESXi complain about a lack of VT-x?

[admin]

gneville wrote:

However I'm having trouble getting 64-bit guests to power on. I keep getting errors such as "longmode is unsupported" or the implementation of vt-x is unsupported.

KVM does not virtualize all of the VT-x features required by ESXi.  The vmware.log file should have a list of the missing features, if you want to take a shot at it!

[gneville]

Damn :smileycry: from other posts I've seen it looks CPUID related.

My vmware.log is attached.