Hello guys good evening, I was encountered this error on my host machine when accessing the vcenter server with vSphere Client (HTML5) - partial functionality, any solution or recommendation to solve my current issues on my vCenter Server 6.5 thank you guys and GOD BLESS US ALL...
I am also having this issue after rebooting VCenter Server. service-control shows the following services as stopped. I am having no luck starting them.
EsxAgentManager VMWareCAMService mbcs vapiEndpoint vmware-autodeploy-waiter vmw
are-imagebuilder vmware-network-coredump vsphere-ui vspherewebclientsvc
I've seen the same issue when trying to browse to VCSA:
after a simple restart of the host/appliance, despite proper graceful shutdown/reboot.
The other symptoms include a blank inventory in vSphere Web Client and (HTML5) vSphere Client (pictured below), even though everything was working fine before the reboot.
Trying the suggestion about restarting the WebClient service:
service-control --stop vsphere-client
service-control --start vsphere-client
doesn't seem to have helped, even after waiting 15 minutes.
So still seeking:
a) a way to fix this, short of rebuilding the VCSA appliance (did that once, worked, then after reboot, problem came right back)
b) a way to prevent this issue from happening in the first place
Hope this information helps.
I haven't worked too hard on this one yet, was frankly hoping somebody else had figured it out by now.
I wonder how prevalent this is. Got nothing on the twitters either:
This issue consistently bit me on 4 fresh builds, and once on upgrade from 6.0U2.
Using MAC address DHCP reservation with proper with forward and reverse lookup and FQDN, but happens even with hard-coded name.
So I haven't really defined the problem yet, nevermind found a solution.
My initial impression was that this error seemed to be related to my shutdown technique that first time this issue hit me, which might be because I hadn't been sure to wait long enough for the VCSA appliance to complete its graceful shutdown. But if that were the case, it would be worrisome that it wouldn't recover.
This issue with VCSA is turned out to be repeatable, even with waiting longer for VCSA to reboot. I'll still want to try this VCSA deploy again with a fixed IP ESXi host and fixed IP VCSA appliance, using no names for anything, and see how that goes, to help rule out DNS/DHCP/etc.
Meanwhile, I noticed that this issue could be related to a Platform Services Controller issue, that is cropping up only after first reboot, for not-yet-determined reasons, something VCSA 6.0U2 wasn't apparently prone to in my same exact network and physical hardware environment.
How did I determine PSC might be at play? The hint was right there, in my browser, when visiting the VCSA vSphere Web Client (where vcsa.lab.localhost is the name of my vcsa appliance):
I notice this error, in the yellow area near the top:
Could not connect to one or more vCenter Server systems:
If I visit that URL, I get:
503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x00007f98680bcd40] _serverNamespace = /sdk action = Allow _port = 8085)
Looking for that error, I find this recently updated KB article that actually lists VCSA 6.5:
That said, the suggested fix doesn't seem to work, since I'm not seeing a "duplicate service endpoint" that the KB article suggests I look for.
There's enough people here that I feel more comfortable sharing this unlisted YouTube video I created, showing my simple recreate procedure, now that I know I'm not alone with this strange Photon OS VCSA 6.5 behavior I'm seeing that wasn't there with SuSE-based VCSA 6.0U2:
This is shared to hopefully help move this along toward resolution more rapidly, hoping to find what's different about my environment than the many successful VCSA deployments out there that handle restarts just fine.
I will hopefully have a chance to try this testing again with fixed IPs this weekend, to avoid DNS/DHCP implication, will see.
Our issue has now been resolved and our web client is now fully functional again.
It appears the following accounts need to be added to "Log on as a Batch job".
GPO Path: Computer Configuration -> Windows Settings ->Security Settings ->Local Policies -> User Rights Assignment ->Log on as a batch job
I think the installer is supposed to add the accounts but if this policy is set by a GPO, it is unable to modify it locally.
I hope this helps some of you.
I seem to have fixed my issue by simply typing in a System name in the dialogue pictured below, instead of leaving it blank, which relied on the DHCP reservation/DNS entry in my router that worked under 6.0U2.
I've now survived 5 ESXi reboots, using ESXi host reboot, which was set to shut down all VMs gracefully, and to restart VCSA automatically.
I will continue to test and reboot, but first, a flight to catch, with my server onboard, of course 😉
Here's how I was doing 6.0U2 or 6.5 installs, choosing DHCP, and leaving System name blank (but setting up a DHCP reservation for the VM's MAC address, and setting up a DNS name, by suspending the VCSA 6.5 appliance moments after the power up button appears, and grabbing the fresh VM's MAC, then resuming)
I don't use DDNS:
but figured I'd try to put in a System name anyway:
Five reboots later, I seem to be good, used to fail on first or second reboot/restart of VCSA.
LWCSteve, Adding those accounts to my GPO did the trick for me!
After applying the GPO I restarted the service with service-control --stop vsphere-client then a start - waited a few minutes, killed a chicken as a sacrifice to VMWare and was able to get into the Web Client.
(I think the sacrifice is optional, not sure if it was needed.)
Thanks so much! I've been at this for over a week!
Thanks for the solution! Works for me on 2012 R2 server with vCenter 6.5 (build 4602587).
I didn't use group policy - we have a group on the server where I had to add all the local users created by vCenter installation.
It's a shame that VMware is creating such rubbish... and from the error message in the browser it's impossible to even get a clue what's the problem. 😞