vMotion from ESXi6.7u1 to ESXi6.5u1

[Akopylov]

Hello there.

I've faced the problem during the vm vMotion from ESXi6.7u1 (10302608) to ESXi 6.5u1 (7388607):

The target host does not support the virtual machine's current hardware requirements.

Use a cluster with Enhanced vMotion Compatibility (EVC) enabled to create a uniform set of CPU features across the cluster, or use per-VM EVC for a consistent set of CPU features for a virtual machine and allow the virtual machine to be moved to a host capable of supporting that set of CPU features. See KB article 1003212 for cluster EVC information.

com.vmware.vim.vmfeature.cpuid.ibpb

com.vmware.vim.vmfeature.cpuid.ibrs

com.vmware.vim.vmfeature.cpuid.ssbd

com.vmware.vim.vmfeature.cpuid.fcmd

com.vmware.vim.vmfeature.cpuid.stibp

Both hosts are equipped by the same CPUs - E5-2658v4.

It seems that EXi6.7u1 abuse some CPU instructions which are not used by ESXi6.5u1, and this problem VM was powered off and on on ESXi6.7u1, so it got some new instructions (I assume so).

However I couldn't find any info about diffrenece between used CPU instructions in ESXi6.7u1 and ESX6.5u1.

I guess, there is no way to move this VM in powered on state, am I right?

[dbalcaraz]

You shouldn't have no problem between these versions as long as the vCenter is in at least the newer version of the ESXi host (6.7u1) and, architecture and model from each ESXi host are the same. Check the architecture not only the model.

[a_p_ · Accepted solution]

This is likely an issue with the physical processor, i.e it's microcode, rather than the ESXi version itself.

Newer ESXi versions come with microcode updates to mitigate CPU issues (e.g. Meltdown, Spectre). Unless you did apply the same microcode to the old system by e.g. a BIOS/Firmware update, the processors will not be compatible without EVC, and VMs which were powered on on the host with the new microcode cannot be live migrated to hosts with older microcode.

André

[BenFB]

The issue is due to the microcode and missing instruction sets. This is directly related to Speculative execution and L1 Terminal Fault. While BIOS updates will add the instruction sets ESXi patches often ship microcode that will do the bare minimum (It's always best to apply platform specific BIOS updates). You need to ensure your host are both patched against the same vulnerabilities and ideally have the correct BIOS version.

VMSA-2018-0004.3

com.vmware.vim.vmfeature.cpuid.ibpb (Indirect Branch Prediction Barrier)

com.vmware.vim.vmfeature.cpuid.ibrs (Indirect Branch Restricted Speculation)

com.vmware.vim.vmfeature.cpuid.stibp (Single Thread Indirect Branch Predictor)

VMSA-2018-0020

com.vmware.vim.vmfeature.cpuid.ssbd (Speculative Store Bypass)

com.vmware.vim.vmfeature.cpuid.fcmd ()