ESX vs. ESXi

[garyfritz]

OK, here are some easy points for someone: Please explain the difference between ESX3.5 and ESXi, and why you would choose one vs. the other. I can't find any comparison. The VMware website basically says "ESXi is the same as ESX, but with a 32MB footprint." I gotta think there's more difference than THAT!

I've been using ESX3.5 to develop some VMs for a class. We need to make sure we have proper licensing for our classroom systems, which sometimes gets messy.

ESXi sounds just like ESX3.5, but it's free. That would simplify life. What would I lose in moving from ESX3.5 to ESXi? Could I export my VMs from ESX3.5 and they'd work no problem in ESXi?

[rriva]

Hi,

it depends very much from which esx 3.5 you are using at this moment.

Assuming you are using (or you plan to use) "foudation license"

The main difference you'll have using esxi instead esx 3.5 are :

• Can't manage multiple host with a same product (Virtual Center)

• Can't use VMware VCB to backup your data (Full VM Backup or File Level Backup)

• Can't use service console (if you have some script or something similar).

and many other.

You can find here a detailed review for differences.

And here another detailed comparison.

Hope this help.

Riccardo Riva

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Thank You!

[garyfritz]

Excellent, that's very helpful. So ESXi has no Linux under it?? Hmm. I end up dropping into Linux fairly often. If ESXi really doesn't need it... I'll look into it.

Thanks!

Gary

[RParker]

ESX is the same across the board. There ARE no Differences. The ONLY exception is ESX 3.5 has a telnet/SSH console. For security reasons, some people CHOOSE to run ESX 3i, but they are identical in EVERY way. ESX 3i is free (basically it's restricted also), however you CAN'T upgrade to ESX 3i, and vice versa. ESX gives no option to go back and fourth, it will format the disk and wipe everything. But other than that they are identical capabilities.

Incidentally ESX is NOT linux, it is NOT built from Linux, the CONSOLE and the boot loader ARE Linux converts (Redhat ES 3 update 9) and that's where the Linux comes in, so you are using a Linux Shell to communicate with the server, but ESX is in no way shape or form Linux. It has SOME components of Linux that you can use to perform SOME functionality, but you should not treat ESX as a Linux server. It performs 1 function, VM Hosting. That's all it should be used for.

So yes ESX 3i is ESX 3.5 with a 32 Meg footprint is a fairly accurate statement.

[Dave_Mishchenko]

It may not be an issue for Gary's implementation, but there are some networking differences that I don't beleive have been resolved - http://kb.vmware.com/kb/1003345.

So yes ESX 3i is ESX 3.5 with a 32 Meg footprint is a fairly accurate statement.

This is an absolutely true marketing statement, but it is only true when one speaks about the "compressed" code for ESXi. Once ESXi is running it is using more space but still not a huge amount compared to ESX and there's definitely a lot less code in ESXi.

Gary, the lack of service console can be a significant issue for some organizations. For example, you can't install hardware monitoring agents on ESXi and thus you have to rely on the CIM providers that come with ESXi. These providers support a limited hardware set and in some case are missing pieces like RAID monitoring.

[Texiwill · Accepted solution]

Hello,

ESX and ESXi are NOT the same. Yes the underlying Hypervisor is the same. Yes the VMFS is the same. Yes the vSwitch is the same, etc. Other than that, they are quite a bit different. Spefically ESXi has a Posix Management Appliance and ESX has a GNU/Linux management appliance. ESXi has no defense in depth. ESX has a defense in depth. When the GNU/Linux Management Appliance crashes, VMs still run. WHen ESXi's management appliance truly crashes (not just daemons going away) then ESXi dies with all its VMs. ESXi is its own beast using its own kernel. ESX uses the vmkernel + another kernel.

These are truly different beasts in many many ways.

Also, ESX is NOT LINUX. LINUX is NOT the underlying component of ESX v3.x. Linux is just the management console for the vmkernel, which is the underlying component. ESXi has expanded the role of the vmkernel to be a relatively general purpose kernel instead of one that just runs VMs, handles resource pools, etc. Some of the vmkernel for ESXi and ESX are the same, but there are some aspects that are quite a bit different.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[garyfritz]

Clearly I was very confused about ESX's design. I thought it was implemented on top of a Linux core. So e.g. I thought I could tweak things in Linux (such as the vmswitch0's IP, or things like that) and ESX "ought" to see those. But if Linux is just kind of "off to the side" of the true ESX kernel, then it's hard telling what you could change in Linux that could affect ESX.

So maybe the things I was using Linux for were wrong anyway. I don't need some of the other Linux features that have been mentioned, so maybe ESXi is a better answer for me than ESX. Thanks all!

[RParker]

From a purely general view ESX are still the same. Under the hood the way they operate, and interact with the hardware may be different, much like a V6 and V8 are different. But both are still gasoline engines, both produce horsepower, both have elements that from a purely topical point of view they ARE the same.

That's what I was referring to. Many people won't care what detailed specs makes them different. ALL they want is for their car to move, cylinders, displacement, that's all good but only for the true mechanic. Same with ESX only a technical person, like yourself, that TRULY understands the makeup, would be interested in what really makes them different. So we can quibble over what really makes them unique, but in the end ALL people want is VM hosting. From the user perspective is anyone REALLY going to notice if you are running a Windows 2003 guest on ESX or ESX 3i? NO. IT just runs, that's all that matters.

So technical differences aside, they are still identical. One has access to the console, the other does not. Everything else (VC, VM, VCB, update manager, etc..) still operate the same, and the end user isn't going to get a different experience. MS Hypervisor and ESX are very different, because they are pretty much NOT compatible, but they do the same thing, they go about it in different ways. The end users can see that difference, but you can't REALLY 'see' the difference in ESX 3.5 or 3i (unless you want to access the console) and for 99% of the people that use ESX that's not really a consideration, they just want VM hosting. ESX 3.5 and 3i have the same capabilities in the end, and that's all we are after.

[Texiwill]

Hello,

ESXi and ESX are pretty much red delicious apples and crab apples. Yes they are both apples but quite a bit different. What is under the hood as you state is important from a security perspective. As is how it is managed. These 'technical issues' as you state are pretty major and glaring differences. They install and run quite a bit differently. I would never call them the same.

As for using ESX being built on top of Linux. Since VI 3.0 has been out that is not true. V2.x was the 'sort of' you describe. The GNU/Linux environment for VI3.x does have access to more things than a normal VM, so what you are trying to do may still work. Not sure what you are trying to do, so its a hard thing to say. If its a common practice there is most likely an esxcfg- style command to enact the action, or some method to do it already.

If you do use ESXi remember it has no real defense in depth so once you break the outer shell, its an open system. I would minimally place its management appliance behind a firewall.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[RParker]

>If you do use ESXi remember it has no real defense in depth so once you break the outer shell, its an open system. I would minimally place its management appliance behind a firewall.

So are you saying that ESX 3.5 w/ SC is more secure than ESX 3i? I was under the impression that without the console things would be more secure, I thought that's why people chose to disdain the console/shell so they run a more pure ESX environment, so ESX 3i isn't as robust security wise?

[Dave_Mishchenko]

Spefically ESXi has a Posix Management Appliance and ESX has a GNU/Linux management appliance. ESXi has no defense in depth. ESX has a defense in depth. When the GNU/Linux Management Appliance crashes, VMs still run. WHen ESXi's management appliance truly crashes (not just daemons going away) then ESXi dies with all its VMs. ESXi is its own beast using its own kernel. ESX uses the vmkernel + another kernel.

In regards to the ESXi management appliance, are you refering to the User World API? If so have you found a way to terminate it or cause it to completely fail? I've had it stop responding and granted that may not constitute a total failure of the API, but the VMs kept running fine. If you mean something else, what component are you referring to? The VMM doesn't seem to have dependecies on the API, although the VMX process does. But from my understanding that would be pretty much the same as with ESX (i.e. the dependence of the VMX process on the service console). And on the topic of the SC are you aware of a way to completely kill it?

If you do use ESXi remember it has no real defense in depth so once you break the outer shell, its an open system.

In terms of security how would you describe this as different from ESX. If you comprimise the SC (and let's say that means root access) then you have complete access.

[-TAZZ-]

In this VMware kb article you will find a comparison between ESX3.5 and ESXi3.5

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100654...

[Texiwill]

Hello,

In regards to the ESXi management appliance, are you refering to the User World API? If so have you found a way to terminate it or cause it to completely fail? I've had it stop responding and granted that may not constitute a total failure of the API, but the VMs kept running fine. If you mean something else, what component are you referring to? The VMM doesn't seem to have dependecies on the API, although the VMX process does. But from my understanding that would be pretty much the same as with ESX (i.e. the dependence of the VMX process on the service console). And on the topic of the SC are you aware of a way to completely kill it?

You are dealing with one kernel on ESXi, this kernel can crash just like any other kernel. If BusyBox crashes yes VMs will stay running but BusyBox is not a kernel just a user land application. ESXi allows the running of generic user processes.

On ESX you are actually dealing with two kernels. If the GNU/LINUX kernel crashes, which is possible, VMs stay running. If the vmkernel crashes, well you have similar failures to ESXi, however you are not running generic user processes within the vmkernel, but tightly controlled VMware only processes.

This has nothing to do with the API in use, but what runs within each kernel in use.

If you do use ESXi remember it has no real defense in depth so once you break the outer shell, its an open system.

In terms of security how would you describe this as different from ESX. If you comprimise the SC (and let's say that means root access) then you have complete access.

There are more than one way to compromise the SC or any management appliance. Root access is just one means. If you gain root access on ANY system but one with Mandatory Access Control, such as SELinux (with proper changes to eliminate root's full control) then all bets are off. However a defense in depth includes proper auditing of commands issues, controlling from where you can access the system, controlling when you can access the system, controlling what a user can see, controlling who can elevate privileges, and controlling what a user can change. In addition, this often means some form of centralized user/password controls are also employed.

The only way to control access to ESXi is to use an external firewall. There is no directory service capability, and no built in 'firewall' or controls for user access outside the normal ones for a Posix environment. In addition there is no capability to audit commands or actions once on the system.

The first thing most people who use ESXi do is break the shell around ESXi by enabling SSH. Not something that is recommended and this can cause all sorts of new issues to pop up. If you do not break the 'shell' then things are better off.

Log into an ESX host some day as a regular user. You will be amazed by the data to which you can gain access. Data that could be used to further pivot attacks or even craft more host specific attacks. Compromised hosts do not necessarily imply just inadvertent root access.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization