VMware Cloud Community
andreaspa
Hot Shot
Hot Shot
Jump to solution

Intel CPU bug - VMware fix on the way?

I've read up on forums, mailing lists and on The Register that there seems to be a severe hardware bug with Intel CPUs:

'Kernel memory leaking' Intel processor design flaw forces Linux, Windows redesign • The Register

There are Linux patches in the works, and Microsoft will release patches during January's patch tuesday. Is ESXi vurnerable, and if so, when can we expect a patch for this? Since it's a critical issue, it will require lots of patching and planning - any heads up would be appreciated!

Tags (2)
1 Solution

Accepted Solutions
wila
Immortal
Immortal
Jump to solution

As far as I have heard so far, the performance hit is in the fixes at the guest OS level, not so much in the hypervisor level fixes.

No idea about performance implications on firmware fixes as intel isn't very communicative on what they did.

edit: see also VMSA-2018-0004  and in particular:

To remediate CVE-2017-5715 in the Guest OS the following VMware and third party requirements must be met:

   

VMware Requirements

  • Deploy the updated version of vCenter Server listed in the table (if vCenter Server is used).
  • Deploy the ESXi patches and/or the new versions for Workstation or Fusion listed in the table.
  • Ensure that your VMs are using Hardware Version 9 or higher. For best performance, Hardware Version 11 or higher is recommended. VMware Knowledge Base article 1010675 discusses Hardware Versions.

Please read the entire article, but the highlighted part is at least about performance

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva

View solution in original post

Reply
0 Kudos
76 Replies
ITaaP
Enthusiast
Enthusiast
Jump to solution

I've been looking into this as well, but haven't seen anything specifically for ESXi. Scary part is there can be up to a 30% performance hit after the update is applied.

https://tactsol.com https://vmware.solutions
Reply
0 Kudos
andreaspa
Hot Shot
Hot Shot
Jump to solution

Yeah, that's one of the scary things - except the possible security issues, of course.. Hope that we'll hear something as soon as the embargo is lifted..

Reply
0 Kudos
Trymon
Contributor
Contributor
Jump to solution

In this case I have also a question.

Would it be enough in this case to patch the Hypervisor or has also every VM to be patched?
I assume it's enough to patch the Hypervisor. If not the Cloud Service providers won't be able to patch their systems.

The performance impact will be a real struggle. Have here some high IO VMs and the performance impact has the most impact if high IO is present.

Reply
0 Kudos
DeepakNegi420
Contributor
Contributor
Jump to solution

VMware has not released any fix at this point. Whilst we wait for VMware to release a patch,  few consideration in planning even patching the linux and windows servers. Obviously, Analyzing the current cluster utilization would be the key to ensure that adequate capacity is available to meet the new demand of 8% to 29% overhead. Also there is a possibility of increased overhead of memory in VMhost if Inter-Virtual Machine Transparent Page Sharing is enabled in VMhosts. It's disabled by default after 5.5 update 2 but check with VMware if you have this enabled.

Regards,

Deepak Negi

Regards, Deepak Negi
Reply
0 Kudos
JakubD
Enthusiast
Enthusiast
Jump to solution

The ESXi 6.0 link from the advisory ( VMware Knowledge Base ) points to patch released back in November - this doesn't seem to be right.

Reply
0 Kudos
JakubD
Enthusiast
Enthusiast
Jump to solution

Also, the patch description under KB 2151132 doesn't mention this CPU vulnerability at all, only OpenSSH, libPNG and network issues.

Reply
0 Kudos
wila
Immortal
Immortal
Jump to solution

Hi,

This bug was kept under NDA so that all the big players could work on it, it was AMD's patch that leaked it. See: Massive security hole in Xeons incoming? - AnandTech Forums

So it is only logical that there is no mention about the problem in the readme on what it fixes as that could possibly leak the issue at hand and would take time away for other players to fix it.

For ex. macOS 10.13.2 released early December has at least some mitigation against it.

and for Linux there was a big patch set on December 4th for exactly this problem, there might have been other patches earlier on, but this is one I found x86/mm: Use/Fix PCID to optimize user/kernel switches · torvalds/linux@6fd166a · GitHub

What I'm trying to say is that it might have been known in November too.

edit: yes this was also known in November, see GitHub - IAIK/KAISER: Kernel Address Isolation to have Side-channels Efficiently Removed  The paper it refers to might have been published before that time. It was presented at a symposium in July.

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
sjesse
Leadership
Leadership
Jump to solution

I think wila is right, the security bulletin refers to   CVE-2017-5753, CVE-2017-5715 and most people other OS vendors are as well for this issue.

Reply
0 Kudos
SteveR123
Enthusiast
Enthusiast
Jump to solution

Hi, did you get a definite response on this ? I'm assuming patching will be required at both hypervisor and guest kernels but would like some confirmation is possible.

Thanks

Steve

Reply
0 Kudos
amitpatel001
Contributor
Contributor
Jump to solution

Hi

Just wanted to know is it fairly easy to patch the hypervisor, i am currently on version 3620759 so not familier in patching esx hosts.

Any pointers on patching really appreciated.

Reply
0 Kudos
sjesse
Leadership
Leadership
Jump to solution

The ticket I opened just reiterated what the security bulletin said, and said they have no more information to provide. If your on 5.5 from what I'm reading CVE-2017-5753 is not fixed but CVE-2017-5715  is

Reply
0 Kudos
sjesse
Leadership
Leadership
Jump to solution

see if you have vmware update manager installed. You upload the patch there and create a "baseline" then attach that to your hosts.

vSphere Update Manager Documentation

Its pretty simple but you want to make sure they are in maintence mode before you do, this patch says it needs a restart.

Reply
0 Kudos
amitpatel001
Contributor
Contributor
Jump to solution

Unfortunatly we dont have update manager installed so i would have to do it the cli way i am afraid.

Reply
0 Kudos
daphnissov
Immortal
Immortal
Jump to solution

amitpatel001​, if you need help patching then open a new thread and folks can assist you. But if you're currently on build 3620759 of ESXi 6.0, then you have more pressing security and stability concerns as you're more than a year-and-a-half outdated on patches.

Reply
0 Kudos
amitpatel001
Contributor
Contributor
Jump to solution

Apologies i will do that. Sorry for problems caused.

Many Thanks

Reply
0 Kudos
wila
Immortal
Immortal
Jump to solution

Hi,

Yes you also need to update your guest OS's as well.

VMSA-2018-0002 - VMware Security & Compliance Blog - VMware Blogs

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
ijwood
Contributor
Contributor
Jump to solution

Hi Wil

That's not quite what the blog entry says though.

It's saying that for the guest vendor OS patch to be effective, you will also have to install the VMWare patch. It doesn't say how effective just installing the VMWare patch by itself will be.

Reply
0 Kudos