VMware Cloud Community
christas
Enthusiast
Enthusiast
Jump to solution

EVC enabled cluster will not allow hosts to rejoin..

Recently I installed a new vCenter Server 5 (the 4.x vCenter was running on a 32-bit OS so upgrade was not an option- fresh install)

I had 1 primary cluster in the datacenter with HA and EVC enabled. All 10 hosts were in it just fine.

However, when the new vCenter Server was up - and after removing the datacenter from the old vCenter, when I went to add the hosts back to the cluster (same EVC mode enabled as before) 6 of the hosts joined just fine, while the other 4 could not.

Has anyone else experienced this?? Just to recap:

new vCenter Server 5

all 10 hosts were in an EVC cluster just fine

4 of the 10 hosts will not rejoin identical cluster...

Thoughts?

VCP5, VCAP5-DCA
Reply
0 Kudos
1 Solution

Accepted Solutions
arturka
Expert
Expert
Jump to solution

Hi Christas,

christas wrote:

One more question regarding EVC - if I point the new vCenter server to the old vCenter's DB, will this cause any issues since there is a new DB created for the new vCenter? How should I approach this?

you can't do it now, you can't connect vCenter 5 to DB in version from vCenter 4, different schema, tables etc. as I stated in my previuos comments, you should migrate to vSphere 5 along with DB

  1. start old vCenter (hopefully you did not remove or disconenct any server from there, right ? )

if you did not made any changes in OLD vCenter, vCenter should be able connect to ESXi hosts back and apply all settings (including EVC)

And yes, I did remove (disconnect the servers from the old vCenter... so I guess reconnecting the old vCenter Server isn't possible.

if you just disconnected (not removed) ESX hosts from old vCenter you should be able connect them back to old vCenter with old settings, but as I said earlier, I don't know what will happed with EVC baselines (because you creted clusters with different EVC on new vCenter.

Artur

VCDX77 My blog - http://vmwaremine.com

View solution in original post

Reply
0 Kudos
21 Replies
a_p_
Leadership
Leadership
Jump to solution

Some details would help.

  • Which EVC mode is configured?
  • What CPU models do you have in the hosts (working and non-working hosts)?
  • Are the hosts in Maintenance Mode when you try to add them to the cluster?
  • Which error message do you get?

André

Reply
0 Kudos
arturka
Expert
Expert
Jump to solution

Hi

Did you migrate DB as well or you made complete new installation of vCetner5 with new DB ?

VCDX77 My blog - http://vmwaremine.com
Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

  • EVC Mode Configured: Intel® "Nehalem" Gen. (Intel ® Xeon® Core™ i7)  for the 1st cluster (6 hosts), Intel® "Westmere" Gen. (Intel ® Xeon® 32nm Core™ i7) for the cluster with the 4 hosts.
  • CPU Models Xeon X5670 @ 2.92 and the other type is Xeon X56XX (I can't remember the last 2 numbers)
  • No the hosts are not in Maintenance Mode when adding them to the cluster (are they supposed to be??)
  • The Error Message is: Incompatible EVC Mode (or something very similar)

Regarding the items I've "forgotten" --- this is actually happening at one of my companies customers network that I've been assigned to assist with...  I will hopefully have better detail next week.

VCP5, VCAP5-DCA
Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

No I did not use the old vCenters DB -- I kept it as a fall back plan if the new vCenter had any issues - So this was a fresh DB created for this machine.

VCP5, VCAP5-DCA
Reply
0 Kudos
Troy_Clavell
Immortal
Immortal
Jump to solution

add the host(s) to the cluster while they are in maintenance mode.

Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

OK. Will this require migration of the running VM's on each host that is to be added back in? 

VCP5, VCAP5-DCA
Reply
0 Kudos
Troy_Clavell
Immortal
Immortal
Jump to solution

...or power off the guests.  A Host cannot enter an EVC cluster if it has guests that are powered on.

Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

I would agree with that, except 6 of the hosts that were joined to the 1st cluster had roughly 10-20 VM's running when joined. As did the other 4 hosts which were put into a seperate cluster under a different EVC mode.

I did read on one of VMware's KB's that virtual guests will have to reboot when changing EVC modes... but the "thought/plan" was that since they were created and running originally on one specific EVC mode, that when being put back into the identical cluster with identical settings, that they wouldn't have to... 

VCP5, VCAP5-DCA
Reply
0 Kudos
Troy_Clavell
Immortal
Immortal
Jump to solution

yes, your logic seems to make sense, but if you are not able to add the host(s) into the cluster, then I don't see any other option then to evacuate the guests currently running on the host, or power of the guests.  Can you vMotion a guest from a host outside of the EVC cluster onto a host in the EVC cluster?

Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

I was able to add the hosts to a separate cluster, just with a different EVC mode enabled.

In order to reboot the guests, a huge change order will have to happen, along with disruption of services that are used for 24/7 dispatching.

I tend to shy away from reboots unless that’s the only option, or something unexpected happens.

I haven’t tried to vMotion the guest between clusters yet, I’ll contact the customer and ask if they have used vMotion between clusters.

VCP5, VCAP5-DCA
Reply
0 Kudos
arturka
Expert
Expert
Jump to solution

christas wrote:

No I did not use the old vCenters DB -- I kept it as a fall back plan if the new vCenter had any issues - So this was a fresh DB created for this machine.

that's why you have a problems now, you should migrate to vcetner 5 together with old DB after migration no problems at all, only few post migration minor tasks

Artur

VCDX77 My blog - http://vmwaremine.com
Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

OK I will try this. The DBA is very protective, but I will see what we can do. Thank you.

VCP5, VCAP5-DCA
Reply
0 Kudos
vGuy
Expert
Expert
Jump to solution

Reply
0 Kudos
christas
Enthusiast
Enthusiast
Jump to solution

Have you ever experienced the problem I've stated above, and migrated the DB's so you can say this is a sure fix? Or is this theory? No offence, but this is a very large network, and I cannot afford to attempt anything that would cause a distruption of services.

Thank you.

VCP5, VCAP5-DCA
Reply
0 Kudos
arturka
Expert
Expert
Jump to solution

Have you ever experienced the problem I've stated above, and migrated the DB's so you can say this is a sure fix? Or is this theory? No offence, but this is a very large network, and I cannot afford to attempt anything that would cause a distruption of services.

Hi Christas,

I will answer your questions another way around, I did many vCenter migrations for past few months, with EVC enabled clusters and never, ever experienced a problems like yours. So, in my opinion you could avoid problems with HW incompatibility If you would migrate vCenter together with DB.

If you want I can do some test in my LAB.

VCDX77 My blog - http://vmwaremine.com
christas
Enthusiast
Enthusiast
Jump to solution

I truly appreciate your offering to do this. The reason we did not initially migrate the 2 DB's was because the DBA has special scripts he has created that generate reports directly from the old vCenters DB tables.

There are a lot of change orders in place that I would have to navigate through in order to migrate the two. However, if it eliminates the issue's we've experienced, then it may come down to migrating the two.

If it fails, do you recommend a way to "go back"? Other than reverting to a previous backup?

VCP5, VCAP5-DCA
Reply
0 Kudos
arturka
Expert
Expert
Jump to solution

I truly appreciate your offering to do this. The reason we did not initially migrate the 2 DB's was because the DBA has special scripts he has created that generate reports directly from the old vCenters DB tables.

this is not a problem, you could migrate only a data from old DB to new DB, the final state would be:

  • old DB still available for rollback or whatever other activities
  • old data and vCenter configuration would be migrated to new DB along with new vCenter server

There are a lot of change orders in place that I would have to navigate through in order to migrate the two. However, if it eliminates the issue's we've experienced, then it may come down to migrating the two.

If it fails, do you recommend a way to "go back"? Other than reverting to a previous backup?

Damn, that hard to say what will happen if you would rollback whole migration because you have enabled new EVC on clusters, different EVC baseline, right ?

But, you could try do something like that:

  1. Disable EVC on new clusters
  2. Disable HA\DRS on new clusters (it will remove FDM agents from ESXi hosts)
  3. disconnect all servers from new vCenter
  4. remove all servers from new vCenter
  5. start old vCenter (hopefully you did not remove or disconenct any server from there, right ? )

if you did not made any changes in OLD vCenter, vCenter should be able connect to ESXi hosts back and apply all settings (including EVC)

I never did such a operation so I can't garant to you that it will succeed.

Artur

VCDX77 My blog - http://vmwaremine.com
christas
Enthusiast
Enthusiast
Jump to solution

One more question regarding EVC - if I point the new vCenter server to the old vCenter's DB, will this cause any issues since there is a new DB created for the new vCenter? How should I approach this?

  1. start old vCenter (hopefully you did not remove or disconenct any server from there, right ? )

if you did not made any changes in OLD vCenter, vCenter should be able connect to ESXi hosts back and apply all settings (including EVC)

And yes, I did remove (disconnect the servers from the old vCenter... so I guess reconnecting the old vCenter Server isn't possible.

VCP5, VCAP5-DCA
Reply
0 Kudos
arturka
Expert
Expert
Jump to solution

Hi Christas,

christas wrote:

One more question regarding EVC - if I point the new vCenter server to the old vCenter's DB, will this cause any issues since there is a new DB created for the new vCenter? How should I approach this?

you can't do it now, you can't connect vCenter 5 to DB in version from vCenter 4, different schema, tables etc. as I stated in my previuos comments, you should migrate to vSphere 5 along with DB

  1. start old vCenter (hopefully you did not remove or disconenct any server from there, right ? )

if you did not made any changes in OLD vCenter, vCenter should be able connect to ESXi hosts back and apply all settings (including EVC)

And yes, I did remove (disconnect the servers from the old vCenter... so I guess reconnecting the old vCenter Server isn't possible.

if you just disconnected (not removed) ESX hosts from old vCenter you should be able connect them back to old vCenter with old settings, but as I said earlier, I don't know what will happed with EVC baselines (because you creted clusters with different EVC on new vCenter.

Artur

VCDX77 My blog - http://vmwaremine.com
Reply
0 Kudos