ESX 3.5 Upgrade Path

[emorgoch]

Hi guys,

I have a pretty good idea of the path to take for this, but I wanted to run it through you guys to see if there's anything we missed. We currently have 3 hosts running ESX 3.0.2, with 39 VMs running across them. VMs are stored on a EqualLogic iSCSI SAN. We also have 2 new hosts that have been rack mounted, and are ready to have ESX installed and added into the cluster. ESX 3 was our first ESX install, so all VMs are ESX 3 format, and the SAN has two LUNs using VMFS3. VirtualCenter runs on a VM on the ESX servers, and the VC database runs on SQL 2000 on a physical machine. DRS and HA are enabled on the single, existing cluster.

After reading through the upgrade guide, the planned upgrade path is as follows:

1. Upgrade VirtualCenter to 2.5, upgrading the VC database as part of the upgrade.

2. Install ESX 3.5 Update 2 on the first new server (ESX4)

3. Add ESX4 to the existing cluster

4. Repeat 2 and 3 on the second new server (ESX5).

5. Put the first host (ESX1) into maintenance mode, wait for the existing VMs to migrate off, and remove the host from the cluster

6. Install ESX 3.5 on ESX1 as a fresh install

7. Add ESX1 back to the existing cluster

8. Repeat 6 - 8 for ESX2 and ESX3

9. Upgrade VM Tools on the existing VMs when able. There is no need to upgrade the VM hardware.

Is there anything that I'm missing? Should we upgrade the existing hosts before adding the new hosts (we currently have n-1 capacity to support it)? Should we be looking at using a second for the upgrade, or is working with everything on the existing cluster fine? Thanks for the help and the peace of mind.

Evan

[JonRoderick · Accepted solution]

Sounds pretty good to me - don't forget the DB backup before you start anything!

You may find that if you add the 3.5 host into your 3.0.2 cluster that VMotion between them isn't automatic - you'll need to carry out manual vmotion events to allow VC to handle the CPU masking changes that may be required. This upgrade path is pretty much identical to the one I've proven in my lab. I don't think you need to upgrade everything before adding the new hosts - they'll give you extra capacity so you don't overload your existing hosts when you remove one to be rebuilt.

Jon.

[lamw]

If you plan on running ESX 3.5 Update 2, you can take advantange of EVC enabled cluster to help auto-configure your hosts to support compatiable VMotion between your different set of hardware and cpu generations. This allieviates the pain of manual masking for every VM when you add a new host with new cpu generation.

[JonRoderick]

I've had trouble getting that working on my setup - documentation is thin on the ground - basically need to enable the VT and NX switches in the BIOS of the host to allow EVC to function - but yes, if your processors support it, it should make the process of setting up masks that much easier.

Jon

[lamw]

It works fine, you just need to have CPU's that support either INTEL-VT or AMD-V, your "nx" bits is what you're masking or presenting from your host OS to your VM, if you look at your VM adv. configuration you'll see those flags, you don't need to touch those to get EVC enabled. EVC is CPU family specific, so if you enable it, you'll do it for either an INTEL cluster or AMD cluster and it'll ensure that when you add future hosts to EVC cluster, that the new server will be VMotion capabitable with the rest of your hosts else it will not allow you to add the hosts. If you mask VMs everytime a new host enters a cluster and when you do cpuid masking, it requires a VM reboot before it takes affect. With EVC, that's all eliminated, you create a new cluster and enable EVC with a blank host, then VMotion from your old cluster to new and then add the old host in until your old cluster becomes your new cluster.

[emorgoch]

With these two servers, we should have enough capacity for the next year or so. And all 5 of the servers run are running the same CPU generation. I'll look it up up EVC though and do some research.

Thanks for the help.