ESX and 64bits guest OS

[cristi4n]

Hi,

Im trying to understand how can the esx servers handle the 64bits guest OS.

If im not mistaken the esx it's a 32bits os (please stop reading if im wrong) hence it has all the memory limitations that they have, specially the kernel virtual space which is going to be always 1gb. That limitation is really annoying for systems with more than 6-8gb of memory cause tries to map in 1gb the 5-7gb of user virtual space. Obviously this doesnt happen in 64bits os cause the limit to the kernel virtual space is the physical ram.

Considering this, it could be the case that a 32bits system with lots of ram (16gb+) is going to crash cause is going to be running out of kernel virtual space (i've seen it happening), so my question is how the esx server handle all this ? and my second questions is, how can the esx run 64bits vms being only a 32bits os ?

cheers,

C.

[RParker]

ESX can virtualize 64-bit OS, by use the Virtual Technology instructions enabled on the CPU from the BIOS. That's how it does it. Yes ESX is 32-bit, but it can run 64-bit inside with no problem.

There are no limitations by doing this. A 64-bit can get it's full memory, but it's limited to 64-gig inside the Guest OS, this is Virtual Limitation, and the ESX server can address up to 256 G on the host (3.5).

[cristi4n]

ok, thanks.

but if the esx is 32 bits the problem with having a small kernel virtual space is still there, meaning that esx servers with more ram, will have more possibilities of failing, since they can run out of memory in the kernel virtual space more quickly..... am i right ?

cheers,

C.

[RParker]

Not really. Somehow VM Ware has been able to overcome the limitation. I am not sure how the process works, I just know that 64-bit guest don't know they are virtualized, and ESX can pass through the VT extensions to give the appear that OS has access to 64-bit processor. It seems to work fine. The memory can be quite high, but you don't want to make VM's too big, if you plan on putting many on one machine.

[cristi4n]

ok, thank you.

it could be great if someone can explain it, cause i've seen servers failing cause they run out of kernel virtual space, i wouldnt like that to happen to my esx server

[jhanekom]

I'm curious to understand why you're concerned about this? Remember that ESX is not a hosted product; the vmkernel runs mostly independent of the Service Console and performs its own memory management. vmkernel is purpose-built and not based on the Linux kernel. There are many techniques that can be used to address more than 4GB in a 32-bit world; PAE is just one of them.

It's definitely an interesting topic that ESX is capable of addressing such vast amounts of memory, but I think I speak for many satisfied customers when I say that ESX won't fall over and die because you've allocated more than 4GB of RAM to VMs! The stated maximum amount of RAM for ESX 3.5 is 256GB. ESX is an enterprise-level product both by marketing and reputation - you can trust it to fulfill that promise.

[cristi4n]

thank you, im sure esx can handle it, im just trying to understand how.

so, i get from your post that processes running on the vms are actually running directly in to the physical ram of the esx server and not on top of it ? hence the only processes that the esx server runs are the processes related to the management of the vms, right ?

[Schorschi]

Hypervisors do not trick the Guest OS, so much as, they simply create separate worlds which are really memory based clones of the hardware BIOS, PCI bus vectors, etc., etc. The reason VMs do not know they are emulated is because there is actually very little difference between BIOS boot strapping in ROM versus RAM. Of course I am glossing over lots of technical issues, but if you think about the fact that the CPU is embedded instructions in structure of a chip, then replicating that code as structure in memory is not that complex. Where it gets crazy is implementing auxiliary functions external to the CPU core implementation. Guest VMs inherit the hardware that is below the VMKernel, considering that the 3i VMKernel which is the core of ESX emulation, is so small it is almost non-existent, it really does beg the question of just how simple the VMKernel is.

[piercelynch]

Well it is work keeping in ming that the management comes from the COS or Service Console.. which is a seperate entity from the Hypervisor. The is what interacts with everything to manage the VMs etc... however, other VMs do not rely on this to operate, so even know that may be 32bit it does not effect the resources available in that manner. The Hypervisor needs to be treated as a completely seperate entity