VMware Cloud Community
COS
Expert
Expert
Jump to solution

ESX 3.x Technical question on x64 Intel Xeons or AMD Opteron.

We will be purchasing a couple of HP Blades for ESX Servers. I have a couple of questions.

Is ESX 3.x compiled in x64 to run "Natively" on 64bit x86 CPU's like the Intel Xeons or AMD Opterons?

If it is not compiled in x64, how does it run x64 guests?

Are there plans for a fully compiled x64 version of ESX?

Thanks

Reply
0 Kudos
1 Solution

Accepted Solutions
oreeh
Immortal
Immortal
Jump to solution

Now what is sending these instructions to the CPU? ESX right?

ESX either sends it to the CPU itself or emulates it (depending on the instruction).

All of these privileged instructions are OS and NOT application related (your SQL server doesn't use Debug registers or HLT the CPU).

And since most 64 bit enhancements only affect register width and operations like add and multiply there's no performance penalty.

The slowest link in the process IMHO is the guest OS (I guess that at least 50% of the W2K3 64bit Server code is still really 32bit).

View solution in original post

Reply
0 Kudos
14 Replies
VirtualNoitall
Virtuoso
Virtuoso
Jump to solution

"Is ESX 3.x compiled in x64 to run "Natively" on 64bit x86 CPU's like the Intel Xeons or AMD Opterons? "

ESX is not itself 64bit

"If it is not compiled in x64, how does it run x64 guests? "

Guests see the native CPU directly. With Intel you need to be sure VT is enabled and the chip has EM64T. IF you are AMD you just need a proc at the right rev.

"Are there plans for a fully compiled x64 version of ESX?"

Only VMware knows for sure.

taylorb
Hot Shot
Hot Shot
Jump to solution

No it runs as a 32 bit OS.

If it is not compiled in x64, how does it run x64

guests?

ESX gives VMs pretty much direct access to the CPU, so they can execute 64 bit instructions. Part of it is the magic that we pay Vmware big $$$ for. It just works is all I know.

Are there plans for a fully compiled x64 version of

ESX?

Haven't heard, but the actual ESX OS doesn't require much juice, so I don't see what the benefit would be.

ThanksWe will be purchasing a couple of HP Blades for ESX

Servers. I have a couple of questions.

Is ESX 3.x compiled in x64 to run "Natively" on 64bit

x86 CPU's like the Intel Xeons or AMD Opterons?

COS
Expert
Expert
Jump to solution

I'm assuming because the guests "see the native CPU directly" there is some sort of software bridge for this to happen. In this case because it is compiled in 32bit it is a 32bit app/bridge to the hardware. Is that correct?

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

Not really, "normal" 64bit operations are totally valid inside a 32bit OS.

Privileged operations (regardless if 32 or 64 bit) are handled the following way (simplified!!!):

Since ESX runs in privileged mode (and uses VT for 64bit VMs) the CPU generates an exception when the VM uses any privileged operations.

This enables ESX to emulate these opeartions.

Unprivileged operations are run directly without intervention from ESX.

Reply
0 Kudos
BUGCHK
Commander
Commander
Jump to solution

The VMkernel can be considered the 'bridge', but it does not emulate all CPU instructions for a VM.

Machine code within the guest VM outside of ring 0 is directly executed by the CPU. Privileged instructions inside the VM are trapped and emulated. That way, the VMkernel has control over the VMs and not the other way round. And the emulation explains why some applications do perform less well in a VM.

I think it does not really matter if the VMkernel is considered a 32-bit or a 64-bit system. There is only one version and it can access more than 4 GigaBytes of memory, even if there are none or only 32-bit guests running.

Reply
0 Kudos
taylorb
Hot Shot
Hot Shot
Jump to solution

My understanding is that VMware acts as a gatekeeper/traffic director to the CPU. So yes, the gatekeeper would be running in 32 bit. Once it gives a thread access to the CPU, that thread can run 64 bit.

Reply
0 Kudos
COS
Expert
Expert
Jump to solution

I don't want to sound like I'm bashing or anything but I need to play the devils advocate and get as much info as I can so when I approach the finances that be, I will be ready to give more of a detailed reason as to why we are going to a 32bit virtualization into the x86 world when we are already deep into the "Native 64bit" virtualization for SQL 2005.

Now, according to what I have heard so far, it sounds more like ESX is doing emulation to execute 64bit guests. So what I am really looking for is someone to say that eventhough 64bit guests are supported it is still being, how can I put this........processed through some sort of 32bit pipe to be executed thus giving the "Emulation"?

I'm getting a headache now...........lol.

Thanks

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

Now, according to what I have heard so far, it sounds more like ESX is doing emulation to execute 64bit guests

no !

only privileged instructions are emulated (it doesn't matter if it's a 32 bit or 64 bit OS)

There's no emulation for other instructions.

An x86 CPU doesn't care if it executes 8, 16, 32 or 64 bit instructions.

If you really want you can execute 64bit instructions in DOS.

Reply
0 Kudos
COS
Expert
Expert
Jump to solution

OK, I see. I read it but went in one eye and out the other.

"Unprivileged operations are run directly without intervention from ESX."

What determines that an operation is Unprivileged or privileged and what is presenting the operation to the CPU?

Bare with me here because I have been out of the VMWare world for almost a year now.

Thanks

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

About 95-99% percent of all instructions are unprivileged.

The CPU determines if a instruction is privileged.

Privileged means that this instruction can ONLY be executed when the executing task operates in ring 0.

Privileged instructions are for example:

CLTS - Clear Task-Switched Flag

LMSW - Load Machine Status

HLT - Halt Processor

LTR - Load Task Register

LGDT - Load GDT Register

LIDT - Load IDT Register

LLDT - Load LDT Register

MOV CRn - Move Control Register

MOV TRn - Move Test Register

MOV DRn - Move Debug Register

IN - Input

INS - Input String

OUT - Output

OUTS - Output String

CLI - Clear Interrupt-Enable Flag (IF)

STI - Set IF

When a VM (which doesn't run in ring 0) executes such an instruction the CPU throws an exception.

ESX intercepts this execption and emulates the instruction or executes it itself.

One example would be a driver accessing a physical port (IN(S), OUT(S)).

Message was edited by:

oreeh

In a VT environment this is a bit different, since some of the privileged instructions are handled / emulated by the CPU itself (LIDT for example).

Most 64 bit enhancements are unprivileged, since these enhancements only affect register width and general operations (add,multiply,...)

Reply
0 Kudos
COS
Expert
Expert
Jump to solution

Perfect. I understand the first part. The CPU determines privileged/unprivileged instructions.

Now what is sending these instructions to the CPU? ESX right?

I'm trying to determine the slowest link in the process before our meeting tomorrow.

Thanks

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

Now what is sending these instructions to the CPU? ESX right?

ESX either sends it to the CPU itself or emulates it (depending on the instruction).

All of these privileged instructions are OS and NOT application related (your SQL server doesn't use Debug registers or HLT the CPU).

And since most 64 bit enhancements only affect register width and operations like add and multiply there's no performance penalty.

The slowest link in the process IMHO is the guest OS (I guess that at least 50% of the W2K3 64bit Server code is still really 32bit).

Reply
0 Kudos
COS
Expert
Expert
Jump to solution

BIG THANKS!

You gave me all the info I needed. Now all I need to do is ask for the $55K for everything.

Thanks again!

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

You are welcome

Now all I need to do is ask for the $55K for everything.

I'm glad that I don't have to help you with this Smiley Wink

Reply
0 Kudos