Hi,
We noticed after a long time, that on one of our vCenter 7.0.3.01000, the service " VMware vStats Service " (vstats) is stopped, while it is supposed to be running as startup is set to " automatic ", no warning in vCenter GUI... 😞
It cannot be started through the GUI (A system error occurred. See logs for details) or even via SSH, the error is the following:
root@xxxxxxxxxxxxxxxxx [ ~ ]# service-control --start vstats
Operation not cancellable. Please wait for it to finish...
Performing start operation on service vstats...
Error executing start on service vstats. Details {
"detail": [
{
"id": "install.ciscommon.service.failstart",
"translatable": "An error occurred while starting service '%(0)s'",
"args": [
"vstats"
],
"localized": "An error occurred while starting service 'vstats'"
}
],
"componentKey": null,
"problemId": null,
"resolution": null
}
The only valuable thing I found support bundle log (xxxxxxxxxxxxxxxxx-2023-06-01--13.53-61238\var\log\vmware\vstats\) is that it does not work anymore since February, 24th, 2023, due to maintenance operation ongoing on that day, the vCenter has been powered off and " VMware vStats Service " (vstats) never started again since then.
I need this service to be running in order to configure backup of this vCenter.
Where can I find useful information that would indicate why this service is not able to run anymore ?
Thanks in advance, any help would be appreciated.
Regards.
Hello,
A solution has been found since last Friday, " vstats " service is still up so far.
The issue comes from the fact that we originally migrated our 6.5 vCenters and external PSC to 7.0 vCenters appliance with PSC embedded.
This is an hypothesis, but we may reasonably think that powering up "accidentally" the old PSC on 02/25 is the root cause of our problem.
Despite the fact that we followed the procedure provided with the 7.0 migration wizard to delete all references to old external PSC, it seems that there were still some kind of link between our vCenters and those supposed decommissioned machines...
The following KB article permits to list references between vCenters and old PSC, it confirms that old PSC are still registered:
https://kb.vmware.com/s/article/2043509
Then, once again we performed the following procedures to decommission old PSC:
https://kb.vmware.com/s/article/75177
https://kb.vmware.com/s/article/2106736
It may be needed to execute this procedure several times if you still find some stale entries of old PSC ...
After that, "vstats" service could be started on vCenter, i hope it helps !
Have you considered updating the VCSA to 7.0.3.01400? I know you are on version 7 but I found an article for 6.5 and 6.7 that references an update that fixed the issue with these version. I'm just wondering if that's been addressed with 7.0.3.01400.
Hello,
Thanks for your reply.
Upgrading is indeed a possibility but that would not explain what the issue was.
I would envisage as well running this tool, maybe I would get interesting information:
https://flings.vmware.com/vsphere-diagnostic-tool
Did you get any resolution on this? I'm running 7.0.3.01400 (21477706) and the same thing happened after using shutdown from the :5480 maintenance page.
Is vstats the only service down?
Did u try "service-control --start --all" ? See which service fails.
Also check logs for /var/log/vmware/vstats/
The issue is not yet fixed, still investigating, I will share here progress of course, if any.
OK, it looks like upgrading to 7.0.3.01400 would not help much in that case ...
Yes, " vstats " is the only service down with automatic startup type.
I did tried to start all services via the usual command line " service-control --start --all ", but it fails on " vstats "...
Like I said in my first post, I have checked \var\log\vmware\vstats\ logs and did not found any information after February, 24th, 2023, when we performed a maintenance operation and the vCenter has been powered off for a duration of around 15 hours.
Which seems like 24th feb something happened... Any logs for those days before it stopped..
In " vstats-stderr.log ", we reapeatdly observe that kind of event:
Just want to add i have the exact same problem...
VCSA 7.0.3.01300
You required open case with vmware
it sees there is missing relations/columns in the database required for vstats service.
@RajeevVCP4 wrote:You required open case with vmware
it sees there is missing relations/columns in the database required for vstats service.
How did you come to this conclusion? Why isn't there a KB for this?
Thanks!
There is internal KB related this error which we can't published here
Hello,
A solution has been found since last Friday, " vstats " service is still up so far.
The issue comes from the fact that we originally migrated our 6.5 vCenters and external PSC to 7.0 vCenters appliance with PSC embedded.
This is an hypothesis, but we may reasonably think that powering up "accidentally" the old PSC on 02/25 is the root cause of our problem.
Despite the fact that we followed the procedure provided with the 7.0 migration wizard to delete all references to old external PSC, it seems that there were still some kind of link between our vCenters and those supposed decommissioned machines...
The following KB article permits to list references between vCenters and old PSC, it confirms that old PSC are still registered:
https://kb.vmware.com/s/article/2043509
Then, once again we performed the following procedures to decommission old PSC:
https://kb.vmware.com/s/article/75177
https://kb.vmware.com/s/article/2106736
It may be needed to execute this procedure several times if you still find some stale entries of old PSC ...
After that, "vstats" service could be started on vCenter, i hope it helps !
Interesting. There is high change, that is the problem as vmdird on 7.0 could get updated again once when the old PSC was powered on. May be some stale entry was left back or a clean decommission was not done.
Thanks for sharing.