Keep getting this error while browsing for vcenter: 503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x0000557fd8e20850] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)
starting manually the vpxd service gives me an error executing start on the service.
Any help with this? Thanks in advance
If you had searched prior to posting (as I advised you earlier), you would have found a thread like this: SOLVED: Could not connect to one or more vCenter Server systems:
Lots and lots of threads with this issue. Please search the forums here.
Check vpxd.log and paste here.
Hi, vpxd.log is almost empty:
2019-11-21 12:35:52 +0200 BEGIN 52103
/etc/init.d/vmware-vpxd status
vmware-vpxd is running
2019-11-21 12:35:52 +0200 END 52103
2019-11-25 16:57:28 +0200 BEGIN 64586
/etc/init.d/vmware-vpxd status
vmware-vpxd is stopped
2019-11-25 16:57:28 +0200 END 64586
But on vpxd-114.log is get these:
2019-11-25T16:22:03.674+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516
2019-11-25T16:22:04.016+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=profileUtil] [DeserializeFromFile] reading data from file: /var/lib/vmware/hpMetadataCache.xml
2019-11-25T16:22:04.016+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=profileUtil] [DeserializeFromFile] reading failed: FileIO error: Could not find file : /var/lib/vmware/hpMetadataCache.xml
2019-11-25T16:22:04.016+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=profileUtil] [LoadCache]: No cache data file found.
2019-11-25T16:22:04.628+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=VpxProfiler] Init [AlarmMoManager::Init()] took 608 ms
2019-11-25T16:22:04.628+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=vpxUtil] [VpxVmomi] Found previous domain socket /var/run/vmware/vpxd-webserver-pipe. Removing...
2019-11-25T16:22:04.628+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for nonChunkingAgents
2019-11-25T16:22:04.628+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for agentsNeedingContentLength
2019-11-25T16:22:04.628+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Max buffered response size is 104857600 bytes
2019-11-25T16:22:04.638+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file or VECS infois not found or is not valid for extension com.vmware.vim.stats.report
2019-11-25T16:22:04.638+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file or VECS infois not found or is not valid for extension com.vmware.vim.vcha
2019-11-25T16:22:04.651+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file or VECS infois not found or is not valid for extension hostdiag
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoDiagnosticMgr] [DiagnosticManagerMo] Running support script located at /bin/vc-support.sh
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to vpxd.usageStats.persist
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to vpxd.usageStats.level
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit
2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to vpxd.usageStats.duration
2019-11-25T16:22:04.761+02:00 error vpxd[7FAAD6D5A700] [Originator@6876 sub=vmomi.soapStub[5]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'
2019-11-25T16:22:04.761+02:00 warning vpxd[7FAAD6D5A700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:04.781+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=licenseServiceDiscovery] Discovered license service endpoint URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:04.781+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=licenseServiceProxy] Using ls URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:04.851+02:00 error vpxd[7FAADD658700] [Originator@6876 sub=vmomi.soapStub[7]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'
2019-11-25T16:22:04.851+02:00 warning vpxd[7FAADD658700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:04.872+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=componentManagerUtil] Discovered VC service folder ID: b91d3f6f-238e-40e0-bdbe-d6310e8633ac
2019-11-25T16:22:04.873+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] Creating SSL Contexts
2019-11-25T16:22:04.873+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Command invoked: /sbin/pidof
2019-11-25T16:22:04.926+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] Finished executing command: /sbin/pidof
2019-11-25T16:22:04.926+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Pid of reverse proxy is: 23720
2019-11-25T16:22:04.933+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=certmgrLogger] Creating CertManagerMo; Mode: vmca
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.* - * requires Global.Settings
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.filesystem* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.filesystem* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.filesystem* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.iscsi* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.iscsi* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.iscsi* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.fcoe* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.fcoe* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.fcoe* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nmp* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nmp* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nmp* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.vm.process.list* - * requires System.View
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.vm.process.kill* - * requires VirtualMachine.Interact.PowerOff
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.license* - * requires Global.Licenses
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.account* - * requires Global.Settings
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.esxcli* - * requires System.View
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.permission* - * requires Global.Settings
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - * requires Host.Config.Settings
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - * requires Host.Config.AdvancedConfig
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.hardware* - * requires Host.Config.Settings
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - * requires Host.Config.Network
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - * requires Host.Config.Storage
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - list* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - get* requires System.Read
2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - * requires Host.Config.Storage
2019-11-25T16:22:04.936+02:00 info vpxd[7FAADD0CD700] [Originator@6876 sub=vpxLro opID=lro-1-56e509fe] [VpxLRO] -- BEGIN lro-1 -- -- VpxdDrmMainLoop --
2019-11-25T16:22:04.936+02:00 info vpxd[7FAADCEC9700] [Originator@6876 sub=vpxLro opID=lro-2-56f32f43] [VpxLRO] -- BEGIN lro-2 -- -- VpxdDrmMainLoop --
2019-11-25T16:22:04.937+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=HostUpgrader] [VpxdHostUpgrader] Could not find vpxa bundle for embeddedEsx version e.x.p: vpx-upgrade-eesx-1-linux-*
2019-11-25T16:22:04.937+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=HostUpgrader] [VpxdHostUpgrader] Could not find vpxa bundle for esx version e.x.p: vpx-upgrade-esx-7-linux-*
2019-11-25T16:22:05.008+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] Wrote vpxd process map to /var/log/vmware/vpxd/vpxd-map-61491.txt.
2019-11-25T16:22:05.009+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Vpxd.MemChecker] vpxd memChecker disabled
2019-11-25T16:22:05.020+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 3726 ms
2019-11-25T16:22:05.020+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key VirtualCenter.MaxDBConnection, pre commit
2019-11-25T16:22:05.020+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key VirtualCenter.MaxDBConnection, pre commit
2019-11-25T16:22:05.020+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to VirtualCenter.MaxDBConnection
2019-11-25T16:22:05.093+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516
2019-11-25T16:22:05.120+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516
2019-11-25T16:22:05.291+02:00 error vpxd[7FAAD77EF700] [Originator@6876 sub=vmomi.soapStub[9]] initial service state request failed, disabling pings. error=HTTP Status:400 'Bad Request'
2019-11-25T16:22:05.291+02:00 warning vpxd[7FAAD77EF700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\Administrator"
2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\com.vmware.vr-38945dd2-22cb-4e7e-a20c-cf8e9141aae9"
2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\HmsAdministrators"
2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\HmsRemoteUsers"
2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\SRM Remote Users"
2019-11-25T16:22:05.308+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] [LazyInitAdmin] Initializing
2019-11-25T16:22:05.308+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] [InitSsoAdminServices]
2019-11-25T16:22:05.308+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] Try to connect to SSO VMOMI endpoint
2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] [InitSsoAdminServices] successful.
2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [CheckTokenValidity]
2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [CheckTokenValidity] Refreshing SSO token ...
2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [RefreshSsoToken]
2019-11-25T16:22:05.382+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [RefreshSsoToken] The VC HOK token has been successfully refreshed.
2019-11-25T16:22:05.417+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] Login successful.
2019-11-25T16:22:05.430+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::SyncClientCache] ignore the exception for adding global permission for user N7Vmacore9Authorize26AuthAlreadyExistsExceptionE(ACE already exists for user 'VSPHERE.LOCAL\Administrator')
--> [context]zKq7AVECAAAAANs6RgAQdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAAZKCAA3WYgAGfOIAG+wgFsaWJhdXRoemNsaWVudC5zbwABGs0BAXXOAQFS0QEChuWgdnB4ZAACCulUAjjyVAJqlVMD4AUCbGliYy5zby42AAJ1jVM=[/context]: VSPHERE.LOCAL\Administrator
2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\Administrators"
2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\SRM-eff48d4d-1ade-4b36-9e06-22b817a4af86"
2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vsphere-webclient-8ed13e22-120c-4b1c-8975-08c637cad0f4"
2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vpxd-8ed13e22-120c-4b1c-8975-08c637cad0f4"
2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vpxd-extension-8ed13e22-120c-4b1c-8975-08c637cad0f4"
2019-11-25T16:22:05.459+02:00 error vpxd[7FAADDC64700] [Originator@6876 sub=vmomi.soapStub[10]] initial service state request failed, disabling pings. error=HTTP Status:400 'Bad Request'
2019-11-25T16:22:05.459+02:00 warning vpxd[7FAADDC64700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:05.487+02:00 info vpxd[7FAADDB62700] [Originator@6876 sub=[SSO]] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrator, false)
2019-11-25T16:22:05.527+02:00 info vpxd[7FAADDB62700] [Originator@6876 sub=[SSO]] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrator, false) res: VSPHERE.LOCAL\Administrator
2019-11-25T16:22:05.527+02:00 warning vpxd[7FAADDB62700] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::SyncClientCache] ignore the exception for adding global permission for user N7Vmacore9Authorize26AuthAlreadyExistsExceptionE(ACE already exists for user 'VSPHERE.LOCAL\Administrator')
--> [context]zKq7AVECAAAAANs6RgANdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAAZKCAA3WYgAGfOIAG+wgFsaWJhdXRoemNsaWVudC5zbwABI9QBAG/tIwBW8CMAe/YrAlR0AGxpYnB0aHJlYWQuc28uMAADLY4ObGliYy5zby42AA==[/context]: VSPHERE.LOCAL\Administrator
2019-11-25T16:22:05.638+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=InvtHostCnx] VpxdInvtHost connect to 3 hosts with 50 workQ
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseClient opID=LicenseClientInit-437266f5] Starting license client
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientInit-437266f5] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Using ls URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientInit-437266f5] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Using ls URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientInit-437266f5] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Using ls URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.655+02:00 info vpxd[7FAAD7E7C700] [Originator@6876 sub=vpxLro opID=lro-3-74856499] [VpxLRO] -- BEGIN lro-3 -- -- QuerySCLRO --
2019-11-25T16:22:05.655+02:00 info vpxd[7FAAD7E7C700] [Originator@6876 sub=vpxLro opID=lro-3-74856499] [VpxLRO] -- FINISH lro-3
2019-11-25T16:22:05.669+02:00 warning vpxd[7FAAD7B76700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:05.670+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Authenticating to LS old API for stubAdapter: [N5Vmomi19SoapStubAdapterImplE:0x00007faac830eea0]
2019-11-25T16:22:05.712+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=StatsAggregator] Jobs not supported by DB. Starting vpxd initiated jobs.
2019-11-25T16:22:05.720+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=drmLogger] [VpxDrmStatsManager::Start] requested rollover hour = 19, currHour = 16
2019-11-25T16:22:05.732+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=OsLayer_linux] [VpxOsLayer] Failed to write to config: FileIO error: Permission denied for file : /etc/vmware-vpx/vpxd.cfg.tmp
2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoServiceInstance] [ServiceInstanceMo::TagServerVm] VirtualCenter Server is running in VM vm-46, tagged as SYSTEM/COM.VMWARE.VIM.VC
2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HTTP server] HTTP server created with docroots /etc/vmware-vpx/sysprep;/etc/vmware-vpx/docRoot;/etc/vmware-vpx/extensions/;/etc/vmware-vpx/locale/, chunked responses enabled: true
2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Http Service started: <server <acceptor p:0x00007faac43eb5d0, h:30, <UNIX '/var/run/vmware/vpxd-webserver-pipe'>>, >
2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Webserver] Web server bound to pipe: /var/run/vmware/vpxd-webserver-pipe
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-1] Created SOAP body handler for vim.version.version11 (vim25/6.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-2] Created SOAP body handler for vim.version.version11 (internalvim25/6.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-3] Created SOAP body handler for vim.version.version10 (vim25/6.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-4] Created SOAP body handler for vim.version.version10 (internalvim25/6.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-5] Created SOAP body handler for vim.version.version9 (vim25/5.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-6] Created SOAP body handler for vim.version.version9 (internalvim25/5.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-7] Created SOAP body handler for vim.version.version8 (vim25/5.1)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-8] Created SOAP body handler for vim.version.version8 (internalvim25/5.1)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-9] Created SOAP body handler for vim.version.version7 (vim25/5.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-10] Created SOAP body handler for vim.version.version7 (internalvim25/5.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-11] Created SOAP body handler for vim.version.version6 (vim25/4.1)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-12] Created SOAP body handler for vim.version.version6 (internalvim25/4.1)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-13] Created SOAP body handler for vim.version.version5 (vim25/4.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-14] Created SOAP body handler for vim.version.version5 (internalvim25/4.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-15] Created SOAP body handler for vim.version.version3 (vim25/2.5u2)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-16] Created SOAP body handler for vim.version.version3 (internalvim25/2.5u2)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-17] Created SOAP body handler for cis.data.provider.version.version1 (cis.data.provider/1.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-18] Created SOAP body handler for cis.data.provider.version.version1 (internalcis.data.provider/1.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-19] Created SOAP body handler for vpx.version.version11 (vpxd3/6.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-20] Created SOAP body handler for vpx.version.version11 (internalvpxd3/6.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-21] Created SOAP body handler for vpx.version.version10 (vpxd3/6.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-22] Created SOAP body handler for vpx.version.version10 (internalvpxd3/6.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-23] Created SOAP body handler for vpx.version.version9 (vpxd3/5.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-24] Created SOAP body handler for vpx.version.version9 (internalvpxd3/5.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-25] Created SOAP body handler for vpx.version.version8 (vpxd3/4.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-26] Created SOAP body handler for vpx.version.version8 (internalvpxd3/4.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-27] Created SOAP body handler for vpx.version.version7 (vpxd3/3.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-28] Created SOAP body handler for vpx.version.version7 (internalvpxd3/3.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-29] Created SOAP body handler for vpx.version.version6 (vpxd3/2.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-30] Created SOAP body handler for vpx.version.version6 (internalvpxd3/2.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-31] Created SOAP body handler for vpx.version.version1 (vpxd3/1.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-32] Created SOAP body handler for vpx.version.version1 (internalvpxd3/1.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-33] Created SOAP body handler for nfc.version.version11 (nfc/6.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-34] Created SOAP body handler for nfc.version.version11 (internalnfc/6.5)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-35] Created SOAP body handler for nfc.version.version10 (nfc/6.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-36] Created SOAP body handler for nfc.version.version10 (internalnfc/6.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-37] Created SOAP body handler for nfc.version.version3 (nfc/3.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-38] Created SOAP body handler for nfc.version.version3 (internalnfc/3.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-39] Created SOAP body handler for nfc.version.version2 (nfc/2.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-40] Created SOAP body handler for nfc.version.version2 (internalnfc/2.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-41] Created SOAP body handler for nfc.version.version1 (nfc/1.0)
2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-42] Created SOAP body handler for nfc.version.version1 (internalnfc/1.0)
2019-11-25T16:22:05.741+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Using default for nonChunkingAgents
2019-11-25T16:22:05.741+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Using default for agentsNeedingContentLength
2019-11-25T16:22:05.741+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Max buffered response size is 104857600 bytes
2019-11-25T16:22:05.742+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter] enableChunkedResponses: true
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Http Service started: <server <acceptor p:0x00007faa983a8bf0, h:49, <TCP '127.0.0.1:8085'>>, >
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] [VpxVmomi] SOAP adapter started on port 8085
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Using default for nonChunkingAgents
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Using default for agentsNeedingContentLength
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Max buffered response size is 104857600 bytes
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter] enableChunkedResponses: true
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Http Service started: [N7Vmacore3Ssl19SSLServerSocketImplE:0x00007faa983d6680]
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] [VpxVmomi] Secure SOAP adapter started on port 8089
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Using default for nonChunkingAgents
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Using default for agentsNeedingContentLength
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Max buffered response size is 104857600 bytes
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Http Service started: <server <acceptor p:0x00007faac409c450, h:23, <UNIX '/var/run/vmware/vpxd-mob-pipe'>>, >
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Vmomi] [VpxdVmomi] MOB started on pipe /var/run/vmware/vpxd-mob-pipe port -1
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /Query
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /ls
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /pcnotification
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /ui
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /vsphere-client
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /vws
2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: pashrepl.pash.gr:8043
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: vcenter.pash.gr:8089
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /Query
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /ls
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /pcnotification
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /ui
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /vsphere-client
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /vws
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: pashrepl.pash.gr:8043
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: vcenter.pash.gr:8089
2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] Proxy HTTP service started
2019-11-25T16:22:05.750+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxdVdb] WarningThreshold: 80% ErrorThreshold: 95%.
2019-11-25T16:22:05.750+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Main] ** BEGIN SERVICES ** (time to active: 4668 ms)
2019-11-25T16:22:05.761+02:00 info vpxd[7FAADCDC7700] [Originator@6876 sub=Heartbeat opID=HeartbeatStartHandler-606df6c8] primary handler started
2019-11-25T16:22:05.787+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Authenticated to LS old API for stubAdapter: [N5Vmomi19SoapStubAdapterImplE:0x00007faac830eea0]
2019-11-25T16:22:05.792+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseClient opID=LicenseClientInit-437266f5] VC license is already assigned
2019-11-25T16:22:05.792+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseClient opID=LicenseClientInit-437266f5] License client started
2019-11-25T16:22:05.871+02:00 error vpxd[7FAADDF6A700] [Originator@6876 sub=vmomi.soapStub[18]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'
2019-11-25T16:22:05.871+02:00 warning vpxd[7FAADDF6A700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:05.890+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Discovered license service endpoint URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.890+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.890+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using ls URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.950+02:00 error vpxd[7FAADC2B1700] [Originator@6876 sub=vmomi.soapStub[20]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'
2019-11-25T16:22:05.950+02:00 warning vpxd[7FAADC2B1700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Discovered license service endpoint URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using ls URL: https://vcenter.pash.gr:443/ls/sdk
2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseAssignmentCleaner opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Run on timer every: 86400 seconds
2019-11-25T16:22:07.049+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=vpxLro opID=2dfc6514] [VpxLRO] -- BEGIN lro-7 -- SessionManager -- vim.SessionManager.loginByToken -- 5297b522-1f4b-91ef-9729-b4adce59f626
2019-11-25T16:22:07.052+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(vsphere.local\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4, false)
2019-11-25T16:22:07.097+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(vsphere.local\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4, false) res: VSPHERE.LOCAL\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4
2019-11-25T16:22:07.097+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=AuthorizeManager opID=2dfc6514] [Auth]: User VSPHERE.LOCAL\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4
2019-11-25T16:22:07.097+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsAdministrators, true)
2019-11-25T16:22:07.115+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsAdministrators, true) res: VSPHERE.LOCAL\HmsAdministrators
2019-11-25T16:22:07.115+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsRemoteUsers, true)
2019-11-25T16:22:07.133+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsRemoteUsers, true) res: VSPHERE.LOCAL\HmsRemoteUsers
2019-11-25T16:22:07.133+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\SRM Remote Users, true)
2019-11-25T16:22:07.151+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\SRM Remote Users, true) res: VSPHERE.LOCAL\SRM Remote Users
2019-11-25T16:22:07.151+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrators, true)
2019-11-25T16:22:07.169+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrators, true) res: VSPHERE.LOCAL\Administrators
2019-11-25T16:22:07.173+02:00 info vpxd[7FAADD1CF700] [Originator@6876 sub=vpxLro opID=15501ba2] [VpxLRO] -- BEGIN lro-8 -- ServiceInstance -- vim.ServiceInstance.retrieveContent -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)
2019-11-25T16:22:07.174+02:00 info vpxd[7FAADD1CF700] [Originator@6876 sub=vpxLro opID=15501ba2] [VpxLRO] -- FINISH lro-8
2019-11-25T16:22:07.174+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=vpxLro opID=2dfc6514] [VpxLRO] -- FINISH lro-7
2019-11-25T16:22:07.181+02:00 info vpxd[7FAADC332700] [Originator@6876 sub=vpxLro opID=72e666b4] [VpxLRO] -- BEGIN lro-9 -- EventManager -- vim.event.EventManager.postEvent -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)
2019-11-25T16:22:07.182+02:00 info vpxd[7FAADC332700] [Originator@6876 sub=vpxLro opID=72e666b4] [VpxLRO] -- FINISH lro-9
2019-11-25T16:22:07.183+02:00 info vpxd[7FAADC230700] [Originator@6876 sub=vpxLro opID=7460ee5a] [VpxLRO] -- BEGIN lro-10 -- ServiceInstance -- vim.ServiceInstance.retrieveContent -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)
2019-11-25T16:22:07.185+02:00 info vpxd[7FAADCF4A700] [Originator@6876 sub=Heartbeat opID=HeartbeatModuleStart-5c8022d1] Certificate not available, starting hostsync for host: host-45
2019-11-25T16:22:07.186+02:00 info vpxd[7FAADC230700] [Originator@6876 sub=vpxLro opID=7460ee5a] [VpxLRO] -- FINISH lro-10
2019-11-25T16:22:07.188+02:00 info vpxd[7FAAD7CF9700] [Originator@6876 sub=vpxLro opID=409434ca] [VpxLRO] -- BEGIN lro-12 -- SessionManager -- vim.SessionManager.logout -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)
2019-11-25T16:22:07.190+02:00 info vpxd[7FAAD7CF9700] [Originator@6876 sub=vpxLro opID=409434ca] [VpxLRO] -- FINISH lro-12
2019-11-25T16:22:07.200+02:00 info vpxd[7FAAD7A74700] [Originator@6876 sub=HostAccess opID=HostSync-host-45-7c09267-TaskLoop-73cbac2a] Using vpxapi.version.version11 to communicate with vpxa at host esxi02.pash.gr
2019-11-25T16:22:08.257+02:00 info vpxd[7FAADDF6A700] [Originator@6876 sub=vpxLro opID=lro-13-66926e8] [VpxLRO] -- BEGIN lro-13 -- -- VsanHostConfigIssuesLRO --
2019-11-25T16:22:08.257+02:00 info vpxd[7FAADDF6A700] [Originator@6876 sub=vpxLro opID=lro-13-66926e8] [VpxLRO] -- FINISH lro-13
2019-11-25T16:22:08.257+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [StringTable::Expand]: nBkts = 256
-->
2019-11-25T16:22:08.325+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [StringTable::Expand]: nBkts = 256
-->
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Execute result code: -1
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] SQL execution failed: INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Execution elapsed time: 1 ms
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Statement diagnostic data from driver is 23505:0:7:ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";
--> Error while executing the query
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Connection diagnostic data from driver is HY000:0:110:
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Bind parameters:
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [0]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 311
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [1]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 4000
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [2]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 1
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [3]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 1
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [4]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 1
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [5]datatype: 11, size: 4, arraySize: 0
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "ok"
2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [6]datatype: 11, size: 20, arraySize: 0
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "VM Network"
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [7]datatype: 11, size: 34, arraySize: 0
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "Network adapter 1"
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [8]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 100
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [9]datatype: 1, size: 4, arraySize: 0
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 7
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [10]datatype: 11, size: 52, arraySize: 0
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "vim.vm.device.VirtualE1000"
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [11]datatype: 11, size: 2146, arraySize: 0
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "<obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:vim25" versionId="6.5" xsi:type="VirtualE1000"><key>4000</key><deviceInfo><label>Netw..."
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [Vdb::IsRecoverableErrorCode] Unable to recover from 23505:7
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [Vdb::IsRecoverableErrorCode] Unable to recover from HY000:110
2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] SQLError was thrown: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";
--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
2019-11-25T16:22:08.392+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=vpxCommon opID=HostSync-host-45-7c09267] [Vpxd_HandleVmRootError] Received unrecoverable VmRootError. Generating minidump ...
2019-11-25T16:22:08.392+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";
--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
2019-11-25T16:22:08.395+02:00 panic vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267]
-->
--> Panic: Unrecoverable VmRootError. Panic!
--> Backtrace:
--> [backtrace begin] product: VMware VirtualCenter, version: 6.5.0, build: build-4602587, tag: vpxd, cpu: x86_64, os: linux, buildType: release
--> backtrace[00] libvmacore.so[0x002B9EA8]: Vmacore::System::Stacktrace::CaptureFullWork(unsigned int)
--> backtrace[01] libvmacore.so[0x001B4E90]: Vmacore::System::SystemFactoryImpl::CreateBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)
--> backtrace[02] libvmacore.so[0x002B7E5B]
--> backtrace[03] libvmacore.so[0x002B7F36]: Vmacore::PanicExit(char const*)
--> backtrace[04] vpxd[0x00722E78]
--> backtrace[05] vpxd[0x00738629]
--> backtrace[06] vpxd[0x0073A837]
--> backtrace[07] vpxd[0x0073FB68]
--> backtrace[08] libvmacore.so[0x0023ED6F]
--> backtrace[09] libvmacore.so[0x0023F056]
--> backtrace[10] libvmacore.so[0x002BF67B]
--> backtrace[11] libpthread.so.0[0x00007454]
--> backtrace[12] libc.so.6[0x000E8E2D]
--> [backtrace end]
If you had searched prior to posting (as I advised you earlier), you would have found a thread like this: SOLVED: Could not connect to one or more vCenter Server systems:
Thanks and sorry, I'll try this and reply if fixed as correct answer.
Good morning,
Thanks for your time and suggestion, the noted article solved my problem and now my vpxd service is up and running after deleting duplicates in postgres VCDB.
Have a nice day