Nested ESXi VMs always on 100 % CPU

[smeagol_2511]

Hello,

Regarding power consumption I would like to run a nested ESXi in Hyper-V on a Windows 10 computer.

This computer has the following configuration :

Hardware

28 GB RAM for the whole computer

240 GB SSD as hard disk

Intel VT and VT-x enabled

Software

Windows 10 1709

Windows Defender disabled (thought that it was the problem but it didn't solve), no antivirus installed

Hyper-V role enabled

Hyper-V VM

One ESXi 6.0 Generation 1 Hyper-V VM with 2 processors and 16 GB RAM. This VM was created  with Legacy Network Adapter Net-Tulip driver imported in ESXi ISO

Nested ESXi hosted VM

In the ESXi I created a Windows 2016 Server VM with 2 processors 4 GB memory and 40 GB Hard disk.

When I booted for installation, it took approximately 8 hours to load the GUI for Windows installation :-s From this step the installation was so quick as a normal computer installation.

After installation, it takes approximatley 4 hours to boot :-s, after this, GUI seems to work correctly. I didn't show during installation, but before  launching GUI, the VM has 100% CPU consummed regarding in the Esxi performance graph :-s

Is there a way to shoot this trouble ?

[smeagol_2511]

If this can help, the second processor is sleeping while the first processor is on 100 % percent, something could be wrong in my processor configuration on Hyper-V ?

[daphnissov]

This is a VMware-focused community and not a Microsoft one. Since your question deals with nested virtualization on Hyper-V, I think you'd have more results either 1) posting on a Hyper-V forum or 2) performing nested virtualization on top of vSphere/ESXi where we can​ be of help.

[smeagol_2511]

OK thanks, gonna try to get an answer from Hyper-V forum, hoping they won't answer me to ask VMWare forum because I use ESXi :-s

[DougBaer]

For what it is worth, running ESXi as a VM, regardless of the underlying hypervisor, is not officially supported by VMware.

Because your base virtualization layer is Hyper-V, this is definitely be more appropriate in a Microsoft forum. In this case, ESXi is the "guest operating system" to Hyper-V.

I suggest looking at any Microsoft documentation regarding running Hyper-V on top of Hyper-V and see what configuration settings they recommend in that case. I have a feeling that that is the best starting point for you.

That said, you are attempting to run a Windows VM on top of an ESXi VM on a Windows Desktop OS. Performance is never going to be great in the second-layer virtual machine: everything must pass through two layers of virtualization. That means quite a bit of overhead for access to memory, disk and network. Even CPU will likely take a hit. Context switching at two layers is not going to result in an optimal user experience. When we do this in our labs, the second-layer VMs are usually something small running Linux (as in, 512 MB RAM, 1 CPU and a few GB disk). They're not running a GUI and only run the minimal services needed to model our use case.

If the goal is to run Windows 2016 in a VM, I'd suggest simply installing the Windows 2016 into a Hyper-V VM on your machine. Adding the extra layer of indirection, while a neat science project, is not likely to get you what you want... unless what you want is to simply show that it can be done.

Good luck

[smeagol_2511]

Hi Doug,

Thank you for detailed answer.

Wow, I have to completely re consider my lab

The goal was to use a 192 GB Proliant ML350 G6 with two processors for nesting 8 ESXi hosting Windows and Linux VM. While load was light enough, I wanted to host the VMs on my desktop computer.

OK, good to know this all before investing money in memory for the Proliant (at the moment it has only 44 GB RAM, I planned to add RAM as one goes along).

Do you think that if I invest in 4 mini ITX PC with 16 GB each and Xeon 1260 processor, using DPM and TPS, could be a solution for testing the following labs  :

          1st lab : shared storage (ONTAP Simulator) on a 5th ESXi for converged storage

          2nd lab : vSAN

Best regards,

[DougBaer · Accepted solution]

If your goal is to host Windows and Linux VMs on a ProLiant that is natively running ESXi, you should be fine. That is something that VMware is very good at.

The way people typically do this (again, not supported) is to run ESXi on the metal and then create ESXi VMs to represent their lab environments.

We call these "vESXi." When running any VM on a vESXi machine, don't expect performance to be *awesome* ... more like "functional" .. and even that depends on the workload and overall load on the hardware. Still, it will generally be a lot faster than hosting your VMs on ESXi in a Hyper-V VM on a desktop machine. When you're doing this, you should consider the speed and number of the disks. Think about how many VMs you will have sitting on the same disk -- most desktop machines have a single drive and you can overwhelm that pretty fast.  I am very rusty on my HP hardware knowledge, but imagine that the ProLiant would have some kind of RAID, or at least more than a single drive.

Beyond the hardware, I suggest taking a look here for some pointers and even pre-built vESXi appliances you can download and use.

https://www.virtuallyghetto.com/nested-virtualization

William has a section on running VSAN in such an environment as well:

https://www.virtuallyghetto.com/vsan

Unfortunately, I am not very familiar these days with what people are doing in home labs, but, as for hardware, you may be interested in this current thread.

https://www.virtuallyghetto.com/2019/01/supermicro-home-lab-group-buy.html

As you can see, there is a ton of information at this site, including using an Apple Mac or an Intel NUC as a host machine, too.

-Doug

https://www.virtuallyghetto.com/2016/03/vsan-6-2-vsphere-6-0-update-2-homelab-on-6th-gen-intel-nuc.h...

[smeagol_2511]

Wow, Awesome !

Big thanks !

This will be very very helpful !