pvevers
Contributor
Contributor

vCenter temporarily disabled/enabled

In VMWare Horizon View Administrator we are currently seeing events being logged continuously relating to vCenter at address being temporarily disabled and almost immediately afterward vCenter at address is enabled.

vCenter at address
${VCAddress} has been temporarily disabled

vCenter at address
${VCAddress} has been enabled

I can find little about these events other than they exist from the documentation library. Are they related to an account problem? If anyone can advise.

Many Thanks

15 Replies
vcpguy
Expert
Expert

I think this is either due to faulty network setting, firewall issue or DNS problem. Could you please check each of these settings on View and VC and get back to us.

----------------------------------------------------------------------------- Please don't forget to reward Points for helpful hints; answers; suggestions. My blog: http://vmwaredevotee.com
0 Kudos
bjohn
Enthusiast
Enthusiast

I've seen this happen occasionally (but not continuously).

I never really found a reason as to why it happens.

0 Kudos
pvevers
Contributor
Contributor

I can confirm I have checked re: the faulty network setting, firewall issue or DNS problem, and confirm these are in order.

0 Kudos
Gaurav_Baghla
VMware Employee
VMware Employee

I have seen this issue in the past and the fix is to restart everything

This is applicable only till 5.1 VMware view and not for 5.2 and higher

To resolve the issue the vCenter server as well as all of the connection servers, including any disabled connection servers, in your environment need to restarted. Security servers and Agent VM’s do not need to be restarted.

The reboot order is as follows:

If Composer is installed on the vCenter server:

  1. Power down all the connection servers in your environment. They must be powered down and not just rebooted. This includes disabled connection servers.
  2. When all the connection servers are powered down, reboot the vCenter server. After reboot ensure that the vCenter services and composer services have come back online.
  3. Once the vCenter server has come back online start the connection servers one at a time. Power on the first server, then wait until the login screen appears before powering on the next server. Order of starting Connection servers does not matter.
  4. Login to View Administrator page and confirm that the issue is resolved.

If Composer is installed on a standalone server:

  1. Power down all the connection servers in your environment. They must be powered down and not just rebooted. This includes disabled connection servers.
  2. When all the connection servers are powered down, shut down the composer server.
  3. When all connection servers and composer server are shutdown,  reboot the vCenter server. After reboot ensure that the vCenter services have started.
  4. Once vCenter server is started and all services are running, start the composer server. Once Composer server reaches the login screen proceed to the next step.
  5. Start the connection servers one at a time. Power on the first server, then wait until the login screen appears before powering on the next server. Order of starting Connection servers does not matter.
  6. Login to View Administrator page and confirm that the issue is resolved.

NOTE: All Connections servers must be started before you can use the View Administrator page. The administrator page may not appear to function until all connection servers are started. This is expected behavior.

Please provide the log snippet from C:\programdata\vmware\vdm\logs

Sort by date and attach the latest file.

Regards Gaurav Baghla Opinions are my own and not the views of my employer. https://twitter.com/garry_14
Gaurav_Baghla
VMware Employee
VMware Employee

Any updates ???????

Regards Gaurav Baghla Opinions are my own and not the views of my employer. https://twitter.com/garry_14
0 Kudos
pvevers
Contributor
Contributor

At the moment the issue is still outstanding.

A full power down cannot take place until the weekend due to the impact it will have on the end users.

This will be done this weekend, and I will update on Tuesday as Friday and Monday are bank holidays.

0 Kudos
vmroyale
Immortal
Immortal

Did you ever get this issue resolved?

Brian Atkinson | vExpert | VMTN Moderator | Author of "VCP5-DCV VMware Certified Professional-Data Center Virtualization on vSphere 5.5 Study Guide: VCP-550" | @vmroyale | http://vmroyale.com
0 Kudos
pvevers
Contributor
Contributor

Yes, my apologies. The resolution as advised by Gaurav_Baghla, rectified the issue. I will update the call accordingly.

0 Kudos
pvevers
Contributor
Contributor

At least I would if I could. I don't have the function to mark the question as answered. I'm assuming because of the age of the call this is not possible.

0 Kudos
Gaurav_Baghla
VMware Employee
VMware Employee

Age shouldn't be a problem but tha should be fine atleast someone looking for a similar issue should be able to follow the steps to fix that. Thank you for the update

Regards Gaurav Baghla Opinions are my own and not the views of my employer. https://twitter.com/garry_14
0 Kudos
steev
Contributor
Contributor

I'm currently facing the same problem, our weekly VDI pool refresh usually (but not always) fails with half the vm's left in "Task halted..." state, and the cause is this VCenter disconnect. At times we have lots of these disconnects that have gone unnoticed, but when they coincide with, or are caused by, the pool refresh it's clearly noticeable.

The "reboot everything" fix mentioned in this thread doesn't seem to help much, the symptoms always return after a short while.

I have a call open with VMware about this.

So far we have:

Moved VCenter, View Connect Server and SQL server vm's to the same host, thus keeping most network traffic within the vSwitch on the host and off of the physical network.

Didn't help.

Changed the power management settings within the Windows Server in each of the above server vm's to "High Performance" (there are known NIC performance problems when Power Management is in 'Balanced' mode and there is high network load).

Didn't help.

Turned off a load of TCP offload settings on the server NIC's

I thought this had helped after a few days with no error, but the problem is back.

I think we have made improvements, because the disconnects now only happen when the pool refresh happens.

This is ongoing.....

0 Kudos
jmclauch
Contributor
Contributor

We started seeing this just this morning; did anyone find a stable solution other that "switch it off and back on again"?

0 Kudos
JSchulz
VMware Employee
VMware Employee

I just witnessed a repeating series of "vCenter at address https://<vcenter FQDN>:443/sdk has been temporarily disabled" and then "vCenter at address https://<vcenter FQDN>:443/sdk has been enabled," while working with a Horizon View deployment, so I thought I'd add a comment to this thread.  We resolved the issue by ensuring that our routing was symmetric.  Once the asymmetric routing issue was fixed up that seemed to resolve the 'temporarily disabled/enabled' issue and allowed us to create desktop pools, replicas, etc.

0 Kudos
Sravan_k
Expert
Expert

I do see this issue today on my view version 7.0, but I am not sure how to fix it.

Thank you,

Vkmr.

0 Kudos
vXav
Expert
Expert

This also worked for us, thanks for the tip.

We run Horizon 7.4 with vCenter 6.5u1 in a cloud pod architectures (2 pods/2 sites) with a vCenter in each site.

What we did to fix the issue:

  1. Disconnect/sign out all the sessions.
  2. shut down all connection servers of both sites.
  3. Reboot the vCenters in both sites (we run them in VCHA pairs):
    1. shut down passive node
    2. shut down active node
    3. shut down witnes
    4. Power on in any order
  4. power on all connection servers (maybe it changed in newer version but 1 by 1 did not work).
  5. Wait for horizon to delete/create vm...
  6. Run viewdbcheck.cmd on a connection server of each site to clean the error VMs.

After that everything went back to normal. We still had a few error VMs in horizon that are not in vcenter. You can either remove them in the ADAM database or wait for them to get cleaned over time.

0 Kudos