Virtualized performance counters are not supported on the host CPU type? (2017, Kaby Lake, MBP)

[lisfolks]

I have a new MacBook Pro (15", 2017). I have moved my VM files from my old MacBook Pro (15" mid-2010). When I try to open any VM on the new MBP, I get the message "Virtualized performance counters are not supported on the host CPU type."

Searching the web, I have found that I can uncheck the two options in Settings, Processors & Memory, Advanced options. These options enable hypervisor applications and code profiling applications, both of which I need and have been using successfully on the mid-2010 MBP. The guest OS' are Ubuntu and MacOS.

What step(s) am I missing to allow these settings to be used in my guest OS'?

[bluefirestorm]

This is the third thread I have read about a problem not able to support virtual performance counters with Kaby Lake processor. First one was with Windows and the other with 2017 iMac.

This is the link to the thread related to the iMac

Fusion - Option to enable code profiling applications fails on iMac 2017 I7 7700K whereas it works o...

I am beginning to think this is either a bug of Kaby Lake processor family or a bug of VMware Fusion/Workstation unable to recognise the VPMC capabilities.

If you don't mind can you attach a vmware.log in a reply?

[lisfolks]

I've attached the logs for both the Ubuntu and MacOS attempts, renamed appropriately.

Since the VMs didn't open, I didn't update the tools. I'm wondering if I need to uncheck the options, update the tools, then re-check the options. I'll try that for grins.

Update: nope, same message after tools update.

[RickShu · Accepted solution]

Hi lisfolks,

Your problem has been reported to Fusion development team, but as far as I know Kabylake is not officially supported in Fusion 8.x.

Regards,

-Rick

[bluefirestorm]

lisfolks,

Interestingly enough, if I mask one of the Fusion VMs with Kaby Lake or Skylake CPU ID and enable code profiling I get the same error. I have Fusion 8.5.7 with a Crystal Well processor with macOS 10.12.6. But if I don't enable the the code profiling, the VM powers up even with CPU masked as Skylake/Kaby Lake and the guest OS isn't tricked into recognising the virtual processor as Kaby Lake or Skylake (i.e. guest VM still sees Crystal Well). If I power up with code profiling enabled without CPUID masking the VM also powers up fine.

Generally it is better to mask to an earlier processor than to a newer processor as presumably the newer processor has the features of the earlier ones but not vice-versa.

(EDIT: In this case, only the CPUID signature is being masked, not the capabilities/features of the CPU).

So perhaps a possible workaround is to mask CPUID signature to an earlier generation processor.

You could try masking the CPUID signature to the guest VM by adding either one of these lines to the vmx configuration file and see whether the VM powers up with code profiling enabled.

Skylake-S (i7-6700HQ used in 2016 15" MacBook Pro)

cpuid.1.eax = "0000:0000:0000:0101:0000:0110:1110:0011"

Broadwell (Denlow) (Broadwell i7 chip used in 2015 21" iMac)

cpuid.1.eax = "0000:0000:0000:0100:0000:0110:0111:0001"

Crystal Well (i7 chip used in 2014/2015 15" MacBook Pro)

cpuid.1.eax = "0000:0000:0000:0100:0000:0110:0110:0001"

[lisfolks]

Thanks for the info, RickShu​. I will be looking forward to a future update that will fix this issue!

[lisfolks]

Thanks for the tip, bluefirestorm​. I'll give that a try as a workaround for now.

I'll update this post to indicate the result.

Update:

bluefirestorm : I edited the .vmx file to use the Crystal Well mask (bolding the change):

    .encoding = "UTF-8"

    config.version = "8"

    virtualHW.version = "12"

    cpuid.1.eax = "0000:0000:0000:0100:0000:0110:0110:0001"

    vcpu.hotadd = "TRUE"

   ... <snip> ...

Didn't work: I still received the same message. The VM is unable to use the virtualization options.

Questions:

1) Is there a specific location within the .vmx file to add this change (ie, did I put the mask in the wrong place in the file)?

2) What would be the .eax and .edx masks for an i7-620M (Arrandale) - the CPU on my mid-2010 MBP where these VMs work fine with the hypervisor options turned on?

[lisfolks]

As an aside, RickShu​, I'm working on projects requiring Docker within my VMs. Docker requires that virtualization be available. Just noting as an example business case for determining priority of this issue, if it helps.

[lisfolks]

I've attached a log of a bootup on the mid-2010 MBP. How would I set up a mask to match this:

Excerpt:

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID vendor: GenuineIntel

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID family: 0x6 model: 0x25 stepping: 0x5

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID codename: Westmere/Clarkdale (Core i7)

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID name: Intel(R) Core(TM) i7 CPU       M 620  @ 2.67GHz

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000000, 0: 0x0000000b 0x756e6547 0x6c65746e 0x49656e69

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000001, 0: 0x00020655 0x01100800 0x029ae3ff 0xbfebfbff

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000002, 0: 0x55035a01 0x00f0b2e3 0x00000000 0x09ca212c

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000003, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000004, 0: 0x1c004121 0x01c0003f 0x0000003f 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000004, 1: 0x1c004122 0x00c0003f 0x0000007f 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000004, 2: 0x1c004143 0x01c0003f 0x000001ff 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000004, 3: 0x1c03c163 0x03c0003f 0x00000fff 0x00000002

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000004, 4: 0x00000000 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000005, 0: 0x00000040 0x00000040 0x00000003 0x00001120

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000006, 0: 0x00000007 0x00000002 0x00000001 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000007, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000008, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 00000009, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 0000000a, 0: 0x07300403 0x00000004 0x00000000 0x00000603

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 0000000b, 0: 0x00000001 0x00000002 0x00000100 0x00000001

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 0000000b, 1: 0x00000004 0x00000004 0x00000201 0x00000001

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 0000000b, 2: 0x00000000 0x00000000 0x00000002 0x00000001

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000000, 0: 0x80000008 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000001, 0: 0x00000000 0x00000000 0x00000001 0x28100800

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000002, 0: 0x65746e49 0x2952286c 0x726f4320 0x4d542865

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000003, 0: 0x37692029 0x55504320 0x20202020 0x4d202020

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000004, 0: 0x30323620 0x20402020 0x37362e32 0x007a4847

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000005, 0: 0x00000000 0x00000000 0x00000000 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000006, 0: 0x00000000 0x00000000 0x01006040 0x00000000

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000007, 0: 0x00000000 0x00000000 0x00000000 0x00000100

2017-08-21T18:35:48.879-06:00| vmx| I125: hostCPUID level 80000008, 0: 0x00003024 0x00000000 0x00000000 0x00000000

[bluefirestorm]

1) Is there a specific location within the .vmx file to add this change (ie, did I put the mask in the wrong place in the file)?

There is no need for a specific location. Just have to make sure that the VM is powered off/not suspended before you edit the Config File. The potential gotchas with using TextEdit to edit the vmx is the "Smart Quotes" and automatic text replacements. (For example: typing "cpuid", it would change it to "cupid"). I turn off the Smart Quotes.

2) What would be the .eax and .edx masks for an i7-620M (Arrandale) - the CPU on my mid-2010 MBP where these VMs work fine with the hypervisor options turned on?

For i7-620M

cpuid.1.eax = "0000:0000:0000:0010:0000:0110:0101:0101"

cpuid.1.ecx = "0000:0010:1001:1010:1110:0011:1111:1111"

cpuid.1.edx = "1011:1111:1110:1011:1111:1011:1111:1111"

But masking the ecx and edx on CPUID leaf 1 is turning on/off features of the CPU to the guest. At least with Windows 10 VMs, using CPU-Z, if I turn off feature such as SSE4, it won't report SSE4.

Did the Skylake and Broadwell not work also? I would have thought Skylake would be something closer to Kaby Lake. FWIW, on Workstation Pro 12.5.x on Windows SkyLake i7-6700HQ powers up the VM even though the code profiling (called virtualized performance monitor counters or VPMC) is enabled.

One way to confirm from the log whether the CPUID signature masking is successful is to look for the respective processor code names: Skylake, Broadwell, Crystal Well.

On the other hand, if what you need is simply virtualization capabilities inside the VM, you would not need the "code profiling" enabled. I have never used "code profiling" feature (e.g. using software such as Intel VTune to capture VM performance counters). But I have used the "virtualization" feature (i.e. running VMs within the VM) which just requires the "Enable hypervisor applications" checked.

[lisfolks]

Just have to make sure that the VM is powered off/not suspended before you edit the Config File. The potential gotchas with using TextEdit to edit the vmx is the "Smart Quotes" and automatic text replacements. (For example: typing "cpuid", it would change it to "cupid"). I turn off the Smart Quotes.

I use Textmate 2.0, an editor for programmers. It doesn't do any of the "word processing" changes, like smart quotes or adding formatting stuff.

cpuid.1.eax = "0000:0000:0000:0010:0000:0110:0101:0101"

cpuid.1.ecx = "0000:0010:1001:1010:1110:0011:1111:1111"

cpuid.1.edx = "1011:1111:1110:1011:1111:1011:1111:1111"

Yeah, I calculated the same. Those values start out looking okay, but then give me a `guestRequiresAVX2noAVX` message. I played around with it all day, but didn't find any combination that solves both problems. And, right, the Skylake and Broadwell options didn't work either. It does seem that it's just a matter of determining which bits to turn on to match the capabilities. However, as you noted, this is changing the guest, not the host. I really need to be able to change the host. (I saw another of your posts mentioning `cpuidMask`. Sadly, that didn't work at all, and I wasn't able to find the correct keyword, if it even still exists for VMWare Workstation/Fusion.)

One way to confirm from the log whether the CPUID signature masking is successful is to look for the respective processor code names: Skylake, Broadwell, Crystal Well.

Yup, I saw those.

This has been a very interesting learning experience! In ESXi, admins can set mask values through the interface for exactly this kind of thing: when a new host processor messes up the VM's needs. It looks like VMware Workstation/Fusion don't have that capability (and what we're trying to do manually here isn't supported by VMware, sigh).

At this point, I guess I need to go back to my old computer, argh, and wait for VMware to do updates for the new processor. Sigh again.

On the other hand, if what you need is simply virtualization capabilities inside the VM, you would not need the "code profiling" enabled.

I use VMware Fusion Pro 8 to run multiple development environments for the work I do. I use Linux, Windows, and Mac guests with software like Docker (which requires virtualization, for example) and Xcode (which provides profiling within MacOS, for example), among other dev tools.

Maybe I'll try some more later in the week, but right now I've got to give up and get some work done... Thanks so much for your suggestions!

[bluefirestorm]

Not that it matters or makes any difference, the AVX feature is bit 28 in the ecx.

cpuid.1.ecx = "0001:0010:1001:1010:1110:0011:1111:1111"

As for ESXi EVC style masking, I would think it is available in Workstation/Fusion but it is just not documented, and since it's not documented, it is not supported formally. The cpuidMask was what I found in some old community post but I cannot find that string at all in the Workstation Pro vmware-vmx.exe file. So that is likely an obsolete method. Even the cpuid.<leaf>.<register> mask is considered old-style and is to be replaced with featMask.evc.cpuid.<attribute/capability>. So to enable AVX would be featMask.evc.cpuid.AVX = "Val:1"

I am not sure if I would count this as progress but I did manage to get rid of the this type of log entry

| vmx| I125: FeatureCompat: No EVC masks.

Instead it would show something like this (depending on what masks I specify, this is masking to the i7-620M CPUID signature)

| vmx| I125: FeatureCompat: EVC masks:

| vmx| I125: Masking Feature: cpuid.STEPPING - Val:5

| vmx| I125: Masking Feature: cpuid.Intel - Val:1

| vmx| I125: Masking Feature: cpuid.MODEL - Val:0x25

| vmx| I125: Masking Feature: cpuid.FAMILY - Val:6

But run into other problems with VPMC capabilities and this was not even with the "Enable code profiling" checked. When I mask to a Skylake processor, VMware Fusion started the VM to the point the capability was listed Skylake microarchitecture but then would abort with an error of invalid mask.

| vmx| I125: Capability Found: vpmc.microarchitecture.skylake = 0x1

| vmx| I125: [msg.featurecompat.invalidmask2] The virtual machine feature mask incorrectly requires 'vpmc.microarchitecture.skylake' = '1'.

Same thing when the CPUID was masked to i7-620M signature.

| vmx| I125: Capability Found: vpmc.microarchitecture.nehalem-0 = 0x1

| vmx| I125: [msg.featurecompat.invalidmask2] The virtual machine feature mask incorrectly requires 'vpmc.microarchitecture.nehalem-0' = '1'.

Maybe I have yet to find the right way to mask the vpmc microarchitecture. But on the other hand, if the workaround requires a CPU VPMC architecture that Fusion recognises (of which Kaby Lake it clearly does not), it is back to square one for this Kaby Lake problem.

In case you want to fiddle with this some time later, the masking method I have done was to create a file named

/Library/Preferences/VMware Fusion/config

The following config file example below masks the CPUID signature to a Skylake.

.encoding = "UTF-8"

featMask.evc.cpuid.Intel = "Val:1"

featMask.evc.cpuid.Family = "Val:6"

featMask.evc.cpuid.Model = "Val:0x5e"

featMask.evc.cpuid.Stepping = "Val:3"

featCompat.evc.completeMasks = "TRUE"

[lisfolks]

So, I played a little more, but no joy. It did take my masks, and the log lists the features. However, it still fails.

I will play more as I have time, bluefirestorm, and update this thread if I get some success; however, I think what RickShu​ was getting at is that underlying code changes need to be made before this processor will work with VMware.

I found recent Intel docs regarding the Kaby Lake processors. They suggest that Intel has changed the way the feature flag information is retrieved in code. That may be why the host masks don't work, even if they're correctly recognized by the VMware software. When it comes time to use the actual info from the processor, the software just isn't getting the right answers from its calls. It's not that the processor doesn't have the needed features; rather it's that the needed features are no longer where VMware expects them to be

I'll hope VMware can get this fixed soon!

[bluefirestorm]

The fact that the current version of VMware Fusion/Workstation identifies it as Pentium 3 code name in the vmware.log suggests it isn't able to handle/support the Kaby Lake processors properly. And the text "Kaby" isn't in the vmware-vmx.exe at all.

You could give the Fusion 2017 Tech Preview a try Fusion Pro Tech Preview on the Kaby Lake Macbook Pro, if you haven't done so, otherwise it just becomes an expensive piece of equipment not used to full potential for a few months. FWIW, in the Workstation Pro 2017 Tech Preview, there are at least 4 different Kabylake codenames.

Kabylake-S/H QS

Kabylake-U/Y QS

Kabylake-S/H ES

Kabylake-U/Y ES

[lisfolks]

Thanks for the tip, bluefirestorm​. I've downloaded the new version and begun testing. I'll move forward with my comments in the Preview community.

[petrosfago]

hello bro i have a problem with vmware can you help me ?

VMware Workstation does not support virtualized performance counters on this host. Refer to VMware KB article 81623 at https://kb.vmware.com/s/article/81623 for more details.

Module 'VPMC' power on failed.

Failed to start the virtual machine.

[ColoradoMarmot]

Workstation doesn't run on Macs.  Please post to the correct forum (this is the one for Fusion).