vSphere 4.1 (ESX) to 5.1 (ESXi) using New Hardware

[AngelC2]

After reading lots of documentation via the Update Center site and some other posts on this community I wanted to see if the plan I have is ok or is there a better way?  I've upgraded 3.5 to 4.1 before but it was using the same hardware...this time around...I'm using new hardware for the new vSphere ESXi & vCenter 5.1 infrastructure.

So, this is what I want to do and hopefully the community's input would help guide me to use the best approach for my needs.

Existing environment consists of:

vCenter 4.1 on a physical Windows Server 2008 R2 server with databases running on a separate physical SQL 2008 R2 cluster.  7 ESX 4.1 host servers.  4 of these hosts are in an EVC compatible cluster, 1 is in a DMZ EVC cluster with another host that will be moved into that cluster after upgrade is completed.  1 is a stand alone ESX 4.1 server that will be decommissioned.  VM's on this server are from our TEST/DEV environment and will be cold migrated over to new 5.1 cluster.  All shared storage (i.e. - datastores running VMFS-3) for existing environment is running on our SAN.

New environment will consist of:

4 new ESXi 5.1 host servers running in a cluster.  2 existing DMZ host servers (old current hardware) **These will be replaced later with 2 new servers identical to the existing new hardware being used.  Migrating vCenter Server to a different host machine following KB 5850444.  vCenter 5.1 on a VM running Windows Server 2012 with databases on a separate physical SQL 2008 R2 cluster.  All shared storage (i.e. - datastores running VMFS-5) for new environment is running on our SAN. 

Here is a summary of my plan:

1 - Install ESXi 5.1 on one of the new host servers

2 - Connect to SAN and Create new VMFS-5 datastores

3 - Create a new vCenter 5.1 VM running on new VMFS-5 using Windows Server 2012 OS pointing to existing databases on my physical SQL 2008 R2 cluster.  Use the temporary Eval license option for new vCenter 5.1.  **Follow KB article 5850444 instructions minus database movement since databases will stay in its existing location

4 - Remove existing ESX 4 host servers from original vCenter 4.1 into new vCenter 5.1 inventory **Not sure about this one since my new vCenter 5.1 will be pointing at the existing database, so wouldn't the ESX 4.1 host automatically show up?

5 - VM's will be moved using Enhanced vMotion to new ESXi 5.1 hosts & VMFS-5 datastores on SAN

6 - ESX 4.1 host will then be removed/retired from vCenter 5.1

7 - Upgrade all licensing to new 5.1 and apply to vCenter 5.1 & ESXi 5.1 hosts

8 - Upgrade VM's (VMware Tools then Hardware to 9) using Update Manager

That's it, any input would be greatly appreciated...thanks.

[admin]

Regarding step 4, if you install vCenter 5,1 pointing to the existing db, then the existing inventory of 4.1 hosts will be present. Precautions: - Donot remove ESXi 4.1 from inventory of vCenter 4.1 since that would mean you cannot add the ESXi back as the VMs are running in EVC mode. - recommend to shutdown vCenter 4.1 during upgrade of vCenter 5.1 VM. - Donot upgrade your existing datastores to vmfs5. - Take a backup of db before doing the upgrade. -  regarding db clustering , i am not quite sure if the clustering needs to be broken during the upgrade Rest is good

[AngelC2]

AakashJ,

Thanks for the input but I'm not sure I understand what you mean by "Donot remove ESXi 4.1 from inventory of vCenter 4.1 since that would mean you cannot add the ESXi back as the VMs are running in EVC mode. - recommend to shutdown vCenter 4.1 during upgrade of vCenter 5.1 VM"?

If I use the same database for vCenter 5.1 and the ESX 4.1 host are available and can be managed by vCenter 5.1 since it's backwards compatible in this area, why would I not want to remove the ESX 4.1 hosts after I've moved the VM's into the new ESXi 5.1 hosts?  I won't be adding the ESX 4.1 host servers back as these are being replaced by the new ESXi 5.1 host servers.  I'm planning on moving VMs off old host to new host and removing these older hosts because I'm going to need the Fibre channel cables to connect another new host to my SAN and I'll then be upgrading the license from 4.1 to 5.1 for these so that I can apply them to the new hosts.  So, basically I'm migrating one host at a time from old to new environment to have zero downtime.

Remember these are ESX not ESXi and it's the cluster that runs in EVC mode which controls compatibility for CPU family differences on the hosts not the VMs.

In regards to recommending that I shutdown vCenter 4.1 during upgrade, ok that seems fine since that's a separate server but not sure its necessary based on what the KB article says?

As for "Donot upgrade your existing datastores to vmfs5".  I wasn't planning on doing this...my existing datastores running VMFS-3 are staying.  I'm creating new VMFS-5 datastores to move the VMs into.  Once all is well and running on new VMFS-5 then I'll remove the older VMFS-3 datastores from my SAN.  I'm definitely taking a backup of the DB before doing the migration if I'm going to use the existing databases just to be safe, but to clarify your question...the SQL cluster does not need to be broken during this migration/upgrade.  The existing databases are already running with vCenter 4.1, so according to the KB article all I have to do is create the new vCenter 5.1 and select the option to "use existing databases".

Again, even if I couldn't do this and had to start from scratch with these databases, I'm ok with that...I have a small environment and it's fairly easy for me to add and configure my hosts & clusters, etc.

[admin]

Everything is fine except the fact that cluster controls the EVC features, but once the hosts are running on EVC , then its the VM that emulates the concerned EVC feature...

[AngelC2]

Never heard of VMs taken on any EVC features, so I'll investigate further.  If that turns out to be a problem then I'll just need to do a cold migration from ESX 4.1 host to ESXi 5.1 so that way upon boot up the VM takes on the new hardware and doesn't go by the previous hosts hardware features, etc.

I would rather just vMotion them into the new host servers, so I'll investigate this further by calling our VMware support to clarify my plan before moving forward with it.

Again, thanks.

[AngelC2]

Ok, I have just confirmed that the new CPU family that I'll have on my ESXi 5.1 host servers is from the "Sandy Bridge" generation which vCenter 5.1 supports as an EVC baseline, but the ESX 4.1 host only support up to the "Westmere" generation which is the generation before it.

My VMs (if they take on the EVC features of the host/EVC cluster they come from even if they are going to be stand alone in the new vCenter 5.1 will not allow me to vMotion them into my ESXi 5.1 host cluster using the "Sandy Bridge" EVC baseline...correct?  I would have to use the lower "Westmere" EVC baseline in order to use vMotion or do cold migrations...correct?  I'm sorry for these "am I correct" questions but just want to be sure even if I'm 99% sure this is correct. 

I guess even if I used the "Westmere" EVC baseline, I could always change it to "Sandy Bridge" EVC baseline after all VMs have been upgraded to new VMware Tools & Hardware 9.

Also, I'm leaning towards starting fresh with new databases which means the existing ESX 4.1 host servers will not be coming across in their EVC cluster to my new vCenter 5.1.  I figured that having a new cluster for my new hardware running ESXi 5.1 would be a much cleaner approach and then vMotion the VMs off the ESX 4.1 hosts onto the new ESXi 5.1 EVC Cluster hosts.

Getting closer to having the planning phase completed for this upgrade/migration to vSphere 5.1, so thanks for all your help! 

[admin]

Westmere is the current option since we dont have sandy Bridge on 4.1.Yes they wont allow to be migrated to a Sandy Bridge EVC cluster...Westmere should be used..You are correct :).. If its a new DB, the ESX 4.1 hosts wont a part of new vCenter 5.1 inventory. Further you might need a downtime again for VMs once everything is upgraded to ESXi 5.1 and you intend to change your VMs to run in Sandy Bridge EVC cluster.. Please ask as many questions as you like.. Remember Answer can be Stupid, but questions can never be.. 🙂

[AngelC2]

I spoke to VMware support yesterday about this and they told me that if I start with a fresh database then the VMs on my existing ESX 4.1 host will not come across with those EVC features once those are in the new ESXi 5.1 vCenter inventory as stand-alone host servers.  I would then be able to vMotion them onto an ESXi 5.1 EVC cluster that's using the Sandy Bridge baseline without any problems.  If I used the existing database then this would not be possible and I would have to use one generation behind (i.e. Westmere) EVC baseline in order to vMotion them into the new ESXi 5.1 EVC cluster.

I will vMotion only the VM to the new ESXi 5.1 host which will be still accessing the VMFS-3 datastores that those VMs machine files are stored.  Once it's on the new host then I will proceed with doing a Cold Migration to the VMFS-5 datastore and finally use Update Manager to upgrade the VMware Tools & Hardware to version 9.  So, for this last part with the VMs I will have some downtime which is fine.

[admin]

Question for you.. How are you going to add your ESX 4.1 host to new vCenter 5.1. I mean the same ESX host running in westmere evc mode with VMs running....

[AngelC2]

No, I'm planning on removing it from the EVC cluster then vCenter 4.1 inventory then add it to vCenter 5.1 as a stand-alone ESX 4.1 host.  From there then I'll vMotion the VMs onto ESXi 5.1 host.  Once I have 2 ESXi 5.1 hosts up & running then I'll put those in their own EVC cluster and from there continue putting the VMs via Cold Migration into that cluster & VMFS-5 datastore as I move forward with migration.

[admin]

Sounds right.. Go on.. 🙂

[Jdeb68]

I'm curious as to your findings in regards to what VMware support explained regarding EVC settings not coming across to the new vCenter Server. I am performing an upgrade from 4.1 to 5.1 with new vCenter Server & DB as well as hardware.

[Michelle50]

Hi

Welcome to the communities.

Below link says its supported application.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100576...

If I have lost confidence in myself, I have the universe against me.

[AngelC2]

It wasn't as clear as they first explained, but this is what happened.  Regardless of what EVC was being used by the older 4.1 hardware it's the VM's you have to worry about...because they do retain the attributes of the EVC mode they were in.  So, I took my new hardware and made a new cluster but since they are identical I did not enable the EVC part of it.  I then had to take each VM and power it down and migrate over to new cluster and then the VM takes on the new features of that cluster.  Only my DMZ cluster had to be setup with an EVC mode since the hardware wasn't identical.

I hope that clears it up, let me know if I can be of more help.  My migration is over and went real smooth once I understood these few things that I had left.