Creating EVC cluster / migrating hosts to it / minimal VM disruption

[jftwp]

I am redesigning my environment somewhat and need a bit of advice. I currently have 2 ESX hosts in a non-EVC cluster and have built a new blade server. The new blade and the 2 hosts will work together in an EVC-enabled cluster. I know that I have to run my EVC cluster in "Intel Xeon Core 2" mode. All 3 hosts will support this mode.

The new / 3rd cluster is sitting in its own vCenter, in an EVC-enabled cluster (in the aforementioned mode). It has no running VMs, but is already configured with networking and storage access that match that of the 2 existing/production hosts, which are currently managed by another vCenter instance (but I'm moving them out of that vCenter instance into this new EVC-enabled one, to join up with the new blade.

All hosts involved are running ESX 3.5 U4. vCenter in any/all instances is 2.5 U4.

BIOS's on the older hosts need to be modified for memory XD in order for them to be used in an EVC cluster, so I know I'll need to shut them down at SOME point to get them into the EVC-enabled cluster.

I have an idea as to how to approach all these requirements, with a big goal being ZERO downtime for the running VMs. Can VMs be migrated into a new, EVC-enabled cluster, coming from an older non-EVC-enabled cluster? Are most all current Intel-based VMs already kinda running in the EVC baseline mode of "Intel Xeon Core 2" or am I just kidding myself and must shut them all down at some point once they've 'entered' an EVC-enabled cluster?

How would you go about this migration between vCenters and Clusters? I think I can use 'Disconnect' from the first vCenter (to at least avoid having to go into Maint Mode and then 'Remove' from the first cluster/vCenter database)... beyond that, I'm looking for the most logical approach to get from 'here' to 'there'. Thanks.

[pomiwi]

Hi there - I guess the two seperate VC throws a spanner in the works I guess what I would try is to add the '3rd' new host to your existing Virtual Centre as a single server outside of your existing cluster. Once you can see it I would perform VMotion of ALL VMs onto this single host, finally I would 'Disconnect' the '3rd' ESX host from your old virtual centre and reconnect it to your new virtual centre. All VMs should continue to run etc...

Hope this helps.

[bulletprooffool]

Moving to an EVC cluster will require your VMS yo be rebooted as they will lose instructions (unless the processors are the same)

Moving hosts from cluster to cluster, or vCentre to vCentre is easy enough - you can simply remove a host from 1 vCentre and add it to the other without interrupting the VMs.

[jftwp]

Thanks... but regarding processors being 'the same', well, that's subjective...

Technically? I guess in my opinion they're different, based on series/model. The old/existing servers are HP Proliant DL360G5's which have x5355 series Intel Xeons in them.

The new blade (BL680 G5) has E7440 Xeon cpu's.

So yeah, in that sense, they're different.

That said, I've added the E7440 into its own/new vCenter with its own/new EVC-enabled cluster, which (since I'm running 3.5 and this is my only option) is configured with EVC baseline of "Intel Xeon Core 2".

THAT said, the DL360G5's that have running VMs on them are quad-core cpu's (x5355), but 'older' than the quad-core (E7440) new blade in the EVC baseline cluster running at a "Intel Xeon Core 2" baseline.

THAT SAID, is a reboot of the running VMs truly/really necessary once introduced into the new EVC cluster? Really? For certain? Any reference material to confirm/verify that?

Again, just trying to reduce/eliminate VM downtime for this necessary migration/redesign of our environment. Thanks.

[Amnexi · Accepted solution]

I see the different vCenters/Clusters problem is solved.

What you need now is to know if you can VMotion between two ESXs with different processors. Before existing EVC, I had... and had NOT some trouble doing that. It's not a problem when DIFFERENT pCPUs have the SAME INSTRUCTION SETS (SSE3, SSSE3, SSSE4, and so on...) Even if the sets are different, I was able to mix them handling CPU MASK options, which are per VM (thing I didn't like... there should be an option to change that on a ESX or even CLUSTER basis!).

I recommend you to test the differences between both ESX's pCPU's instructions sets with this tool, I think it's the best out there:

If the instructions sets are different you have a problem, because you would need to find which bits to change in the CPU MASK options in every VM... and that needs reboot, too. I think they surely will be different, because newer pCPUs with FlexMigration (Intel's EVC compatible processors), just because they are new they surely have a new instruction set (maybe SSSE4)

Good luck!

[pfuhli]

This worked for me pretty well

Regards,

daniel

[jftwp]

Here's what I ended up doing.

This allows for

zero downtime.

Old/Source cluster:

1. UNconfigure HA (disable HA on

the source cluster).

2. Select one of the hosts (we'll call it 'HostA' and 'Disconnect' it,

followed by Remove (this avoids maintenance mode, leaves all VMs

running/intact, and allows true 'removal' from vCenter).

New/Target cluster:

1. Highlight datacenter-level and

select Add Host. Use wizard to 'add' HostA to this vCenter instance /

data center.

2. Vmotion HostA's VMs into the

new EVC-enabled host within new cluster. This now houses the VM/s that were previously running on HostA.

3. Reboot HostA and connect to

it via iLo (HP Proliant). Enter BIOS, and change options for both Intel memory

protection and Intel VT (enable)---this is required before moving this host into the

EVC-enabled cluster (for EVC compatibility). Without those settings

confirmed, you cannot add any server hardware into an EVC-enabled cluster.

4. After HostA has booted, drag it

into the EVC-enabled cluster, and check/verify HA and DRS settings on the

cluster. Verify that vmotion works between HostA and any hosts already running in the EVC-enabled cluster.

[MikeS1983]

I was able to do something similar on one of my clusters. Zero downtime.. The CPU's in this cluster were Intel Zeon x5350 and x5365. Stepping levels were 7, 10 and 11. When the new host arrived it had a CPU of Intel Zeon x5450 stepping level 10. After EVC was enabled Vmotion worked fine.

However my other cluster is not having the same luck. They are all X5450 and will soon be bringing in an older 5350. Because of this I'm enabling EVC on this cluster. Hovever when I try to repeat what I did on my first cluster it throws up a CPU incompatability error when trying to Vmotion.

I have downtime scheduled to enable EVC but I just find it weird that this doesn't work the same as the other cluster.

Any ideas?

Mike