Hyper-Threading on ESX Server

Hyper-Threading on ESX Server

Introduction

Those of us at VMware that regularly engaged with our field or directly with customers are often asked how Hyper-Threading impacts the performance of a system. I've now been asked this question enough times to have my canned "It Depends" response on the tip of my tongue for every conference I present at. I'm going to use this document to elaborate on that point a bit and provide a little more detail.

What Is Hyper-Threading?

Hyper-Threading is a technology included by Intel first in their Netburst line of parts. Hyper-Threaded processors present their individual processing cores to the system as if they are two processing cores. To use Intel's parlance, that means that each physical core appears in the operating system as two logical cores. While the OS can distinguish between a system that has two logical cores (i.e. a single physical core with Hyper-Threading enabled) and two physical cores, applications cannot. It is up the the OS's scheduler to choose if it wishes to use logical cores in the same manner as physical cores.

Is It Supported In ESX Server?

Hyper-Threading (HT) has been supported in ESX Server since version 2. ESX Server's scheduler is aware of the presence of HT and treats logical cores differently from physical cores. Virtual CPUs (vCPUs) requesting resources are assigned first to physical cores until all physical cores are loaded. If there are additional vCPUs requesting CPU resources they will then be assigned to the additional logical cores. By this method HT has no impact on performance until more vCPUs are concurrently executing than there exist physical cores.

How Does It Perform on ESX Server?

Understanding HT performance on native systems is tricky enough. Try Googling "hyperthreading performance" and you'll discover a world of information on this feature. By faking the presence of another processing core, Hyper-Threading removes some SMP scheduling from the guest operating system which is then handled by the processor's thread scheduler. Since the processor can manage context switches between its threads much faster than the OS can, this means that often heavy parallelism in applications results in improved performance due to Hyper-Threading.

The exact gains due to HT even on native systems is dependent on the workload. The industry has cited numbers that range from 0% to 40% gain when HT is enabled on supported processors. For the most part, HT improves performance nominally. There are a few cases where performance can slow down, but these are exceptions rather than the norm.

The best generalizations we can provide about HT on ESX are:

  1. Until you have more vCPUs requesting processing power than there are physical cores, HT cannot hurt and provides no value.

  2. Once you have more vCPUs requesting CPU than physical cores on the system, HT usually provides small gains.

  3. While very early versions ESX may have sub-optimally handled HT, since ESX Server 2.5.3 robust support of HT in the scheduler means that it should not hurt performance.

For More Information

Hyper-Threading configuration options for performance optimization were provided in ESX Server 3 and beyond. See the Resource Management Guide (page 123) for more information.

Comments

Does the (potential) vulnerability discussed in the KB below still exist for ESX/ESXi 4.0 on Intel 5500 (Nehalem) processors?

Security Response to CERT VU#911878: Possible Information Leak with Hyper-Threading

http://kb.vmware.com/kb/1728

Thanks.

Bump...

This comment is based on my own performance measurements using the passmark tool.

Hyper-Threading gives you a 15% to 20% performance gain on "common" VM's. Some CPU intensive applications which are not multi-threaded build can suffer badly from Hyper-Threading functionality. I think this is because a physical CPU core is split and delivers 50% of the perfomance of the physical core on a vCPU (1 CPU with 4 cores and Hyper-Threading enabled gives you 8 vCPU's).

Because the application is not multi-threaded it will not make full use of multi-threading. Disabling Hyper-Threading on the VM gives a slight improvement. Disabling Hyper-Threading in the BIOS of the ESX host makes the performance go up to around 170%.

Best regards,

Roderick

htttp://www.roderick-ict.nl

Version history
Revision #:
1 of 1
Last update:
‎05-09-2008 02:31 PM
Updated by: