Nested hypervisor support under VBS (inc. Device Guard)

[ralish]

Seeing as the survey links are broken, at least for me (see here), I'm posting on the board instead.

A feature I haven't seen discussed but which would be extremely useful is nested hypervisor support under Hyper-V enabled hosts (i.e. using the Windows Hypervisor Platform). I've posted some thoughts about this before here, but to summarise, if running on a host which is Hyper-V enabled you can't run guests under VMware Workstation which expose Intel VT-x/EPT. I assume the same issue is present if exposing AMD-V/RVI but don't have such a system to test on. Virtualising the IOMMU does work.

The impact is you can't run nested virtualisation scenarios on a system with Hyper-V enabled, be it because you actually use Hyper-V alongside VMware Workstation, or it's a dependency of other features like Device Guard. Where this is particularly frustrating is it blocks running VBS enabled guests as they require VT-x/AMD-V.

This limitation doesn't appear to apply to Hyper-V itself, as such configurations work fine on Hyper-V VMs, which suggests it's technically possible.

[wila]

---

@ralish wrote:

This limitation doesn't appear to apply to Hyper-V itself, as such configurations work fine on Hyper-V VMs, which suggests it's technically possible.

---

Technically possible for Hyper-V does not mean that Microsoft has made a programming interface available to do so from a programming point of view.

From what I understand so far, that is what is holding back VMware from having this feature. Without MS exposing this, VMware cannot provide it. It's a chicken & the egg problem.

AFAIK, there is no nested virtualisation possible when going through the Windows Hypervisor Platform API's, only when using VMware's hypervisor or -like you say- using Hyper-V itself directly.

Perhaps this has changed with the new Workstation Tech Preview, but I've not read anything about that so far.

--
Wil

[ralish]

Looking at the Windows Hypervisor Platform headers it does appear to be possible.

There's the NestedVirtSupport bit of the WHV_PROCESSOR_FEATURES1  structure which is passed as the input buffer to WHvSetPartitionProperty with the WHvPartitionPropertyCodeProcessorFeaturesBanks property code. There's also the WHvPartitionPropertyCodeNestedVirtualization property which appears to take a BOOL as the input buffer to the function.

I'm not clear how these two approaches differ, or how one affects the other. The WHvPartitionPropertyCodeNestedVirtualization property feels the most promising. It's noted in the Data Types documentation for the function that NestedVirtualisation is supported since Windows 10 19H2.

This is from a very quick look at the API documentation and header files, so may not be 100% accurate, but overall appears promising.