ESXi 3.5 U3 installation within workstation 6.5 and server 2.0

[hjdohrmann]

Setting up an ESXi server within a VM seems to be a good approach for testing and playing arround, I thought...

Compared to looking for a cheap HW to set up a physical ESXi it should be much easier, I thought...

I am a newby with ESXi...

Looking arround on VMware communities and the internet it turns out, that there seems to be as much confusion

about how to setup a VM as about how to setup a physical server with ESXi!

What I think to know about the VM approach is the following (may be helpful for other ones...):

• Setting up ESXi as VM seems to be possible with some tips&tricks, tweaking vmx file, etc.

• VMware Workstation 6.0 and 6.5 as well as VMware server 2.0 should support this approach,
  there seem to be a few VMware gurus around, who got it running...

• It should work on Intel and AMD 'newer' dual core CPUs,
  we should have 64bit support, HW virtualization support and DEP (HW data execution prevention)

• We should have a 64 bit OS, where we install Server 2.0 or Workstation 6.x, but this is only a requirement, when we plan to run 64 bit guests within, right?
  ESXi from my understanding is not a64 bit application but a 32 bit, right?
  So there seems to be no need for a 64 bit OS on our machine.
  If we do so, we must know, that we only can setup 32 bit OS in our ESXi server. But this is suffient for testing...

So far so good, let's do it. I will start with the VMware workstation 6.5 I just got.

My platform for testing:

• AMD Athlon64 X2 5200+, 8GB RAM, ASUS M2N mainboard, SATA disks, etc.

• Windows Server 2008 SP1 Data Center Edition

• VMware Workstation 6.5.1 build-126130

What I have done then:

• Setup a new VM in VMware workstation with these settings:
  - Guest OS: other
  - 2 CPUs
  - 1048 MB RAM (have seen postings saying 1GB is not sufficient, need a bit more than 1GB)
  - VMware HW version: 6.5
  - 10GB Disk, SCSI, persistent, not preallocated (you may also use preallocated, if you have sufficient free space)
  - 1 NIC bridged
  - 3 NICs custom (VMnet3), for future use...

• CD/DVD(IDE): connected to ISO: F:\VMware6.5-VMs\VMware-VMvisor-InstallerCD-3.5.0_Update_3-123629.i386.iso

• Did not start the VM now but , check/add something to the created vmx file:
  - Add the following:
  monitor.virtual_exec = "hardware"
  monitor_control.enable_svm = "TRUE" (for AMD, but this seems to be an old setting no more necessary for ESXi 3.5 U3 ???, not sure so left it in)
  monitor_control.restrict_backdoor = "TRUE"
  
  If you have INTEL CPU use:
  monitor_control.vt32 = "TRUE" (for Intel but this seems to be an old setting no more necessary for ESXi 3.5 U3 ???)
  
  - Check the following lines and correct if necessary:
  scsi0.virtualDev = "lsilogic"
  ethernet0.virtualDev = "e1000"
  ethernet1.virtualDev = "e1000"
  ethernet2.virtualDev = "e1000"
  ethernet3.virtualDev = "e1000"

• Then powered up the VM:
  - Installation started from ISO
  - And now it goes very slowly...
  after about 3 or 4 h :smileyalert: this typical yellow screen disappears and a black one comes up!
  - [F11] shows some strange error messages, see attached screenshot
  - Installation seems to hang now...
  - Oh no, it's just going on a bit (the initial installation screen is comming up again (Welcome... Please select CANCEL / REPAIR / INSTALL)
  Obviously the VM has restarted but there is no bootable installation on the disk to boot ???
  
  This is at least the 5th or sixth try to get it running, very frustrating!
  Have not expected that after all these good and helpful hints I found before.
  
  Question: why does it need nearly 4 hours and then ands up with a non bootable installation???

Does anybody have any idea what is going wrong???

Is it Workstaiton 6.5, which causes the problem,

Are I doing something stupid ot totally wrong???

What are these error messages saying (see attached)?

Obviously they have come up at the very beginning of this installation and then it has gone on.

Tomorrow will try the same within server 2.0...

Any help appreciated,

Hans

[hjdohrmann]

Sorry, obviously I was a bit confused before ,when saying:

'the initial installation screen is coming up again (Welcome... Please select CANCEL / REPAIR / INSTALL)

Obviously the VM has restarted but there is no bootable installation on the disk to boot ??? '

No, this is not the initial screen, it is "the installation screen", what means the VM has needed about 4 hour just to to do the preparation (HW detecting, etc.) for the installation and now it has started the installation...

Anyway this is very strange for me, installation is now also so slow (2% progress in the last 15 minutes...)

Need help, to understand what is going wrong!

[admin]

With Workstation 6.5, you need to specify the following settings in the configuration file for the ESX VM:

monitor.virtual_exec = "hardware"
monitor_control.restrict_backdoor = "TRUE"

The first option enables VT-x or AMD-V support, depending on chip vendor. Note that Workstation 6.5 only supports AMD-V on AMD Family 10H or later processors.

It sounds like you are using an AMD Family 0FH processor. Workstation 6.5 does not support AMD-V on Family 0FH processors, so you are not going to be able to run ESXi 3.5 in a VM on this hardware.

[hjdohrmann]

Thanks a lot for this hint.

So my settings in the vmx file seems to be ok, the old one (monitor_control.enable_svm = "True" is no more necessary).

Ok, the CPU:

• I am bit confused about the HW requirements for Workstation 6.5.

• In the user manual of workstation 6.5 (p. e3) there is only stated, that for 64 bit support a AMD CPU Revision D or later is required.

• My CPU is:Athlon64 X2 5200+ (sorry not 5000, as I said before),
  CPU-Z gives the following information:
  - AMD Athlon64 X2 5200, 2,7 GHz, Brisban, 65nm, 1,296 V
  Brand ID: 4, Family: F, Model: B, 6B: Revision BH-G2
  2x 64 KB Level1 Cache, 2x 512KB Level2 Cache, 64Bit,
  
  ??? Is that a Revison "G2", "BH" or "F" compared to the terminology VMware is using in the maunal???

How does these numbers match with what you said before:

• Workstation 6.5 does not support AMD-V on Family 0FH processors,
  It must be at least 10H !?
  Don't understand exactly which processors you mean (what is a 10H processor?)

• Are these numbers really related to Athlon64 X2 Dual Core procs or do you mean Opterons ???

Could you give me some recomendations which Athon64 X" procs or Intel Core 2 Duo/Quad procs are supported by

Workstation 6.5 or Server 2.0?

Or where to find this information? Haven't found anything in the VMware documentation!

Any help much appreciated!

[admin]

It sounds like you have a Family 0FH Rev G CPU. As indicated in the documentation, 64-bit guests are supported on this CPU.

However, although this CPU does offer AMD-V, Workstation 6.5 does not support AMD-V on this CPU. There is a particularly egregious bug in the AMD-V implementation on Family 0FH processors for which there is no software workaround in Workstation 6.5. Note that AMD-V is not required for 64-bit guest support. See KB 1901 for details.

AMD-V (or VT-x) is required for running nested VMs, or even for simply running ESX in a VM. See . Note that neither running nested VMs nor running ESX in a VM is supported.

Workstation 6.5 and Server 2.0 support AMD-V on all Family 10H AMD64 processors (these CPUs are commonly referred to as Barcelona or Phenom). Workstation 6.5 and Server 2.0 support VT-x on all currently shipping Intel EM64T processors that have VT-x.

[wila · Accepted solution]

What forgets to tell you is that the workaround for the 0F version of the AMD processor is still available in workstation 6.0.x

It has been removed for ease of code maintenance and well.. the processor won't be around for ever..

So you should be able to get ESX running under Workstation 6.0.x

Another hint is that the family numbers of processors are written out in hexadecimal notation not in decimal as humans like to use...

We programmers tend to forget that you might not know that 0F (hex) = 15 (dec) and that 10 (hex) = 16 (dec)

So version 10 (H) is just family version 16 and you are still on 15.

--

Wil

[hjdohrmann]

Thanks a lot for the clarification regarding processor numbers, family, revision, etc.

Now I understood what you mean.

So I have the following options to get the nested VM running:

- Switch back to Workstation 6.0 on this AMD platform with Family "F" CPU (special thanks to Wil for this useful hint)

- Change the platform to AM2+ board with AMD Phenom proc and go on with Workstation 6.5

- Change the platform to Intel (any Core 2 Duo /Quad proc)

what I have done in the meantime is:

- setup a 2nd test system based on Intel Core 2 Duo E8400 and now it works without any issues!

Could easily install ESXi as VM in Workstation 6.5 within about 5 minutes!

Hopefully this experience would help others to get ESXi running as a VM.

Hans

[hjdohrmann]

Wil, thanks for your hint regarding using workstation 6.0 instead of 6.5 to get it running!

I will check it also.

What about Server 2.0? Obvoiusly these family F AMD procs are also not supported?

Hans

[wila]

Hi,

Thanks for the points.

What about Server 2.0? Obvoiusly these family F AMD procs are also not supported?

I'm not sure if it works on Server 2.0, there's a reasonable chance it does, but Server certainly isn't the same animal as Workstation. I believe that ESXi does work under VMware Fusion 2.0 (Mac OS X) which I personally find a lot more useful The VMware team has made special changes on request of the community fo get this support in Workstation and I'm not sure if that code has already made it into VMware server or if it will make it into Server. Someone else will have to jump in on this I'm afraid.

If it works in Server 2.0 then it won't work with AMD family F processors as the Virtual hardware is the same generation as with workstation 6.5.

--

Wil

[admin]

I don't believe ESXi 3.5 runs under Workstation 6.0 due to a bug in that release of Workstation.

Workstation 6.5, Player 2.5, Server 2.0, and Fusion 2.0 all have roughly comparable capabilities for running nested VMs and for running ESX in a VM. There is no support for AMD-V on Family 0FH processors in any of these products.

[wila]

Thanks Jim, that's good to know. You can have my points now 🙂

--

Wil

[admin]

So I have the following options to get the nested VM running:

- Switch back to Workstation 6.0 on this AMD platform with Family "F" CPU (special thanks to Wil for this useful hint)

- Change the platform to AM2+ board with AMD Phenom proc and go on with Workstation 6.5

- Change the platform to Intel (any Core 2 Duo /Quad proc)

Option 1 will not work, since ESXi 3.5 does not run under Workstation 6.0. (At least, you cannot launch nested VMs.)

Be careful of option 3, since there are some Core 2 CPUs that do not have VT-x.

Before you make any hardware investments, let me remind you that this is not a supported configuration. ESX is not a supported guest under Workstation, nor is the Workstation VM on the ESX Hardware Compatibility List.

[pnavahal]

check this post might be help full for using intel core 2 duo

http://communities.vmware.com/thread/154182

[hjdohrmann]

Thanks for this link, but unfortunately can not access, get:

Unauthorized

You do not have the permissions to perform this action or access this page. For further assistance, please contact a VMware Communities moderator

I am aware of the fact the some Intel Dual Core processors are not supporting Intel-VT, like:

- some Core Duo: T2450, T2250, T2050, T2300E

- some Core2Duo: E7200, E7300, E7400, E8190, E4300 to 4700, T5200 to T5550

- Core2Quad: only Q8200

- all Pentium Dual Core

Hans

[hjdohrmann]

Jim, thanks a lot for your many useful hints and tips!

Think I will go on with he Intel platform on my E8400 system. As I said before, I got ESXi running without any issues within a few minutes...

Since all this is only for testing and learning, I don't see any problems that Workstation is not an officially supported platform for ESX/ESXi.

[TomHowarth]

That is expected as it is a pointing to the WS6.5 beta forums,

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth

VMware Communities User Moderator

Blog: www.planetvm.net

[Marooned]

I would like to state that this did work for me using VMware Workstation 6.0 an AMD DualCore +6000 A2 cpu with a Biostar A2 (not +) but there are several things that I did differently. The workstation version shouldn't matter as long as you follow step 2.

1. When you created the vm, make sure you right-click and Remove from Favorites. If you do not, the changes that you make in the .vmx file will not stay, they will return back to the default settings.

2. Select Custom for the vm type, but select Workstation 5 then select ESX Server check box.

3. Do make sure that Virtualization is enabled in the bios.

4. The Operating system I chose in the installation was RedHat Enterprise Linux 5 x64.

I got it installed in under two minutes.

These settings also work to install ESX 3.5 Server

Hope this helps