vcenter appliance 6.5.0 vpxd service not starting

[oBull]

Keep getting this error while browsing for vcenter: 503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x0000557fd8e20850] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

starting manually the vpxd service gives me an error executing start on the service.

Any help with this? Thanks in advance

[daphnissov]

Lots and lots of threads with this issue. Please search the forums here.

[KocPawel]

Check vpxd.log and paste here.

[oBull]

Hi, vpxd.log is almost empty:

2019-11-21 12:35:52 +0200 BEGIN 52103

/etc/init.d/vmware-vpxd status

vmware-vpxd is running

2019-11-21 12:35:52 +0200 END 52103

2019-11-25 16:57:28 +0200 BEGIN 64586

/etc/init.d/vmware-vpxd status

vmware-vpxd is stopped

2019-11-25 16:57:28 +0200 END 64586

[oBull]

But on vpxd-114.log is get these:

2019-11-25T16:22:03.674+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516

2019-11-25T16:22:04.016+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=profileUtil] [DeserializeFromFile] reading data from file: /var/lib/vmware/hpMetadataCache.xml

2019-11-25T16:22:04.016+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=profileUtil] [DeserializeFromFile] reading failed: FileIO error: Could not find file  : /var/lib/vmware/hpMetadataCache.xml

2019-11-25T16:22:04.016+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=profileUtil] [LoadCache]: No cache data file found.

2019-11-25T16:22:04.628+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=VpxProfiler] Init [AlarmMoManager::Init()] took 608 ms

2019-11-25T16:22:04.628+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=vpxUtil] [VpxVmomi] Found previous domain socket /var/run/vmware/vpxd-webserver-pipe. Removing...

2019-11-25T16:22:04.628+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for nonChunkingAgents

2019-11-25T16:22:04.628+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for agentsNeedingContentLength

2019-11-25T16:22:04.628+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Max buffered response size is 104857600 bytes

2019-11-25T16:22:04.638+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension com.vmware.vim.stats.report

2019-11-25T16:22:04.638+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension com.vmware.vim.vcha

2019-11-25T16:22:04.651+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension hostdiag

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoDiagnosticMgr] [DiagnosticManagerMo] Running support script located at /bin/vc-support.sh

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to vpxd.usageStats.persist

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to vpxd.usageStats.level

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit

2019-11-25T16:22:04.695+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to vpxd.usageStats.duration

2019-11-25T16:22:04.761+02:00 error vpxd[7FAAD6D5A700] [Originator@6876 sub=vmomi.soapStub[5]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2019-11-25T16:22:04.761+02:00 warning vpxd[7FAAD6D5A700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:04.781+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=licenseServiceDiscovery] Discovered license service endpoint URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:04.781+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=licenseServiceProxy] Using ls URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:04.851+02:00 error vpxd[7FAADD658700] [Originator@6876 sub=vmomi.soapStub[7]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2019-11-25T16:22:04.851+02:00 warning vpxd[7FAADD658700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:04.872+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=componentManagerUtil] Discovered VC service folder ID: b91d3f6f-238e-40e0-bdbe-d6310e8633ac

2019-11-25T16:22:04.873+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] Creating SSL Contexts

2019-11-25T16:22:04.873+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Command invoked: /sbin/pidof

2019-11-25T16:22:04.926+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] Finished executing command: /sbin/pidof

2019-11-25T16:22:04.926+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Pid of reverse proxy is: 23720

2019-11-25T16:22:04.933+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=certmgrLogger] Creating CertManagerMo; Mode: vmca

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.* - * requires Global.Settings

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.filesystem* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.filesystem* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.filesystem* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.iscsi* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.iscsi* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.iscsi* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.fcoe* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.fcoe* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.fcoe* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nmp* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nmp* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nmp* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.vm.process.list* - * requires System.View

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.vm.process.kill* - * requires VirtualMachine.Interact.PowerOff

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.license* - * requires Global.Licenses

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.account* - * requires Global.Settings

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.esxcli* - * requires System.View

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.permission* - * requires Global.Settings

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - * requires Host.Config.Settings

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - * requires Host.Config.AdvancedConfig

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.hardware* - * requires Host.Config.Settings

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - * requires Host.Config.Network

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - * requires Host.Config.Storage

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - list* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - get* requires System.Read

2019-11-25T16:22:04.934+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - * requires Host.Config.Storage

2019-11-25T16:22:04.936+02:00 info vpxd[7FAADD0CD700] [Originator@6876 sub=vpxLro opID=lro-1-56e509fe] [VpxLRO] -- BEGIN lro-1 --  -- VpxdDrmMainLoop --

2019-11-25T16:22:04.936+02:00 info vpxd[7FAADCEC9700] [Originator@6876 sub=vpxLro opID=lro-2-56f32f43] [VpxLRO] -- BEGIN lro-2 --  -- VpxdDrmMainLoop --

2019-11-25T16:22:04.937+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=HostUpgrader] [VpxdHostUpgrader] Could not find vpxa bundle for embeddedEsx version e.x.p: vpx-upgrade-eesx-1-linux-*

2019-11-25T16:22:04.937+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=HostUpgrader] [VpxdHostUpgrader] Could not find vpxa bundle for esx version e.x.p: vpx-upgrade-esx-7-linux-*

2019-11-25T16:22:05.008+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] Wrote vpxd process map to /var/log/vmware/vpxd/vpxd-map-61491.txt.

2019-11-25T16:22:05.009+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Vpxd.MemChecker] vpxd memChecker disabled

2019-11-25T16:22:05.020+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 3726 ms

2019-11-25T16:22:05.020+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key VirtualCenter.MaxDBConnection, pre commit

2019-11-25T16:22:05.020+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key VirtualCenter.MaxDBConnection, pre commit

2019-11-25T16:22:05.020+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to VirtualCenter.MaxDBConnection

2019-11-25T16:22:05.093+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516

2019-11-25T16:22:05.120+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516

2019-11-25T16:22:05.291+02:00 error vpxd[7FAAD77EF700] [Originator@6876 sub=vmomi.soapStub[9]] initial service state request failed, disabling pings. error=HTTP Status:400 'Bad Request'

2019-11-25T16:22:05.291+02:00 warning vpxd[7FAAD77EF700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\Administrator"

2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\com.vmware.vr-38945dd2-22cb-4e7e-a20c-cf8e9141aae9"

2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\HmsAdministrators"

2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\HmsRemoteUsers"

2019-11-25T16:22:05.306+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\SRM Remote Users"

2019-11-25T16:22:05.308+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] [LazyInitAdmin] Initializing

2019-11-25T16:22:05.308+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] [InitSsoAdminServices]

2019-11-25T16:22:05.308+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] Try to connect to SSO VMOMI endpoint

2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] [InitSsoAdminServices] successful.

2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [CheckTokenValidity]

2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [CheckTokenValidity] Refreshing SSO token ...

2019-11-25T16:22:05.341+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [RefreshSsoToken]

2019-11-25T16:22:05.382+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][SsoAdminFacadeImpl]] [RefreshSsoToken] The VC HOK token has been successfully refreshed.

2019-11-25T16:22:05.417+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=[SSO][AdminAdapter]] Login successful.

2019-11-25T16:22:05.430+02:00 warning vpxd[7FAAEC88F800] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::SyncClientCache] ignore the exception for adding global permission for user N7Vmacore9Authorize26AuthAlreadyExistsExceptionE(ACE already exists for user 'VSPHERE.LOCAL\Administrator')

--> [context]zKq7AVECAAAAANs6RgAQdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAAZKCAA3WYgAGfOIAG+wgFsaWJhdXRoemNsaWVudC5zbwABGs0BAXXOAQFS0QEChuWgdnB4ZAACCulUAjjyVAJqlVMD4AUCbGliYy5zby42AAJ1jVM=[/context]: VSPHERE.LOCAL\Administrator

2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for group "VSPHERE.LOCAL\Administrators"

2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\SRM-eff48d4d-1ade-4b36-9e06-22b817a4af86"

2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vsphere-webclient-8ed13e22-120c-4b1c-8975-08c637cad0f4"

2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vpxd-8ed13e22-120c-4b1c-8975-08c637cad0f4"

2019-11-25T16:22:05.431+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=AuthorizeManager] [ACL] Adding unresolved permission for user "VSPHERE.LOCAL\vpxd-extension-8ed13e22-120c-4b1c-8975-08c637cad0f4"

2019-11-25T16:22:05.459+02:00 error vpxd[7FAADDC64700] [Originator@6876 sub=vmomi.soapStub[10]] initial service state request failed, disabling pings. error=HTTP Status:400 'Bad Request'

2019-11-25T16:22:05.459+02:00 warning vpxd[7FAADDC64700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:05.487+02:00 info vpxd[7FAADDB62700] [Originator@6876 sub=[SSO]] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrator, false)

2019-11-25T16:22:05.527+02:00 info vpxd[7FAADDB62700] [Originator@6876 sub=[SSO]] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrator, false) res: VSPHERE.LOCAL\Administrator

2019-11-25T16:22:05.527+02:00 warning vpxd[7FAADDB62700] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::SyncClientCache] ignore the exception for adding global permission for user N7Vmacore9Authorize26AuthAlreadyExistsExceptionE(ACE already exists for user 'VSPHERE.LOCAL\Administrator')

--> [context]zKq7AVECAAAAANs6RgANdnB4ZAAAkJ4rbGlidm1hY29yZS5zbwAAhE8bAOqXGAAZKCAA3WYgAGfOIAG+wgFsaWJhdXRoemNsaWVudC5zbwABI9QBAG/tIwBW8CMAe/YrAlR0AGxpYnB0aHJlYWQuc28uMAADLY4ObGliYy5zby42AA==[/context]: VSPHERE.LOCAL\Administrator

2019-11-25T16:22:05.638+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=InvtHostCnx] VpxdInvtHost connect to 3 hosts with 50 workQ

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseClient opID=LicenseClientInit-437266f5] Starting license client

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientInit-437266f5] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Using ls URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientInit-437266f5] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Using ls URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientInit-437266f5] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.638+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Using ls URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.655+02:00 info vpxd[7FAAD7E7C700] [Originator@6876 sub=vpxLro opID=lro-3-74856499] [VpxLRO] -- BEGIN lro-3 --  -- QuerySCLRO --

2019-11-25T16:22:05.655+02:00 info vpxd[7FAAD7E7C700] [Originator@6876 sub=vpxLro opID=lro-3-74856499] [VpxLRO] -- FINISH lro-3

2019-11-25T16:22:05.669+02:00 warning vpxd[7FAAD7B76700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:05.670+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Authenticating to LS old API for stubAdapter: [N5Vmomi19SoapStubAdapterImplE:0x00007faac830eea0]

2019-11-25T16:22:05.712+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=StatsAggregator] Jobs not supported by DB. Starting vpxd initiated jobs.

2019-11-25T16:22:05.720+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=drmLogger] [VpxDrmStatsManager::Start] requested rollover hour = 19, currHour = 16

2019-11-25T16:22:05.732+02:00 error vpxd[7FAAEC88F800] [Originator@6876 sub=OsLayer_linux] [VpxOsLayer] Failed to write to config: FileIO error: Permission denied for file  : /etc/vmware-vpx/vpxd.cfg.tmp

2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoServiceInstance] [ServiceInstanceMo::TagServerVm] VirtualCenter Server is running in VM vm-46, tagged as SYSTEM/COM.VMWARE.VIM.VC

2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HTTP server] HTTP server created with docroots /etc/vmware-vpx/sysprep;/etc/vmware-vpx/docRoot;/etc/vmware-vpx/extensions/;/etc/vmware-vpx/locale/, chunked responses enabled: true

2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=HttpSvc.HTTPService] Http Service started: <server <acceptor p:0x00007faac43eb5d0, h:30, <UNIX '/var/run/vmware/vpxd-webserver-pipe'>>, >

2019-11-25T16:22:05.732+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Webserver] Web server bound to pipe: /var/run/vmware/vpxd-webserver-pipe

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-1] Created SOAP body handler for vim.version.version11 (vim25/6.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-2] Created SOAP body handler for vim.version.version11 (internalvim25/6.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-3] Created SOAP body handler for vim.version.version10 (vim25/6.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-4] Created SOAP body handler for vim.version.version10 (internalvim25/6.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-5] Created SOAP body handler for vim.version.version9 (vim25/5.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-6] Created SOAP body handler for vim.version.version9 (internalvim25/5.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-7] Created SOAP body handler for vim.version.version8 (vim25/5.1)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-8] Created SOAP body handler for vim.version.version8 (internalvim25/5.1)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-9] Created SOAP body handler for vim.version.version7 (vim25/5.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-10] Created SOAP body handler for vim.version.version7 (internalvim25/5.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-11] Created SOAP body handler for vim.version.version6 (vim25/4.1)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-12] Created SOAP body handler for vim.version.version6 (internalvim25/4.1)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-13] Created SOAP body handler for vim.version.version5 (vim25/4.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-14] Created SOAP body handler for vim.version.version5 (internalvim25/4.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-15] Created SOAP body handler for vim.version.version3 (vim25/2.5u2)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-16] Created SOAP body handler for vim.version.version3 (internalvim25/2.5u2)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-17] Created SOAP body handler for cis.data.provider.version.version1 (cis.data.provider/1.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-18] Created SOAP body handler for cis.data.provider.version.version1 (internalcis.data.provider/1.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-19] Created SOAP body handler for vpx.version.version11 (vpxd3/6.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-20] Created SOAP body handler for vpx.version.version11 (internalvpxd3/6.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-21] Created SOAP body handler for vpx.version.version10 (vpxd3/6.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-22] Created SOAP body handler for vpx.version.version10 (internalvpxd3/6.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-23] Created SOAP body handler for vpx.version.version9 (vpxd3/5.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-24] Created SOAP body handler for vpx.version.version9 (internalvpxd3/5.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-25] Created SOAP body handler for vpx.version.version8 (vpxd3/4.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-26] Created SOAP body handler for vpx.version.version8 (internalvpxd3/4.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-27] Created SOAP body handler for vpx.version.version7 (vpxd3/3.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-28] Created SOAP body handler for vpx.version.version7 (internalvpxd3/3.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-29] Created SOAP body handler for vpx.version.version6 (vpxd3/2.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-30] Created SOAP body handler for vpx.version.version6 (internalvpxd3/2.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-31] Created SOAP body handler for vpx.version.version1 (vpxd3/1.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-32] Created SOAP body handler for vpx.version.version1 (internalvpxd3/1.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-33] Created SOAP body handler for nfc.version.version11 (nfc/6.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-34] Created SOAP body handler for nfc.version.version11 (internalnfc/6.5)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-35] Created SOAP body handler for nfc.version.version10 (nfc/6.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-36] Created SOAP body handler for nfc.version.version10 (internalnfc/6.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-37] Created SOAP body handler for nfc.version.version3 (nfc/3.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-38] Created SOAP body handler for nfc.version.version3 (internalnfc/3.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-39] Created SOAP body handler for nfc.version.version2 (nfc/2.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-40] Created SOAP body handler for nfc.version.version2 (internalnfc/2.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-41] Created SOAP body handler for nfc.version.version1 (nfc/1.0)

2019-11-25T16:22:05.737+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SOAP-42] Created SOAP body handler for nfc.version.version1 (internalnfc/1.0)

2019-11-25T16:22:05.741+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Using default for nonChunkingAgents

2019-11-25T16:22:05.741+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Using default for agentsNeedingContentLength

2019-11-25T16:22:05.741+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Max buffered response size is 104857600 bytes

2019-11-25T16:22:05.742+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter] enableChunkedResponses: true

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SoapAdapter.HTTPService] Http Service started: <server <acceptor p:0x00007faa983a8bf0, h:49, <TCP '127.0.0.1:8085'>>, >

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] [VpxVmomi] SOAP adapter started on port 8085

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Using default for nonChunkingAgents

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Using default for agentsNeedingContentLength

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Max buffered response size is 104857600 bytes

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter] enableChunkedResponses: true

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=SSL SoapAdapter.HTTPService] Http Service started: [N7Vmacore3Ssl19SSLServerSocketImplE:0x00007faa983d6680]

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] [VpxVmomi] Secure SOAP adapter started on port 8089

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Using default for nonChunkingAgents

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Using default for agentsNeedingContentLength

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Max buffered response size is 104857600 bytes

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=DebugBrowser.HTTPService] Http Service started: <server <acceptor p:0x00007faac409c450, h:23, <UNIX '/var/run/vmware/vpxd-mob-pipe'>>, >

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Vmomi] [VpxdVmomi] MOB started on pipe /var/run/vmware/vpxd-mob-pipe port -1

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /Query

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /ls

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /pcnotification

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /ui

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /vsphere-client

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: /vws

2019-11-25T16:22:05.743+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: pashrepl.pash.gr:8043

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Default] New/Updated endpoint added: vcenter.pash.gr:8089

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /Query

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /ls

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /pcnotification

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /ui

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /vsphere-client

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: /vws

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: pashrepl.pash.gr:8043

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] ProcessFile: Value of Servernamespace is: vcenter.pash.gr:8089

2019-11-25T16:22:05.744+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=MoReverseProxy] Proxy HTTP service started

2019-11-25T16:22:05.750+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=vpxdVdb] WarningThreshold: 80% ErrorThreshold: 95%.

2019-11-25T16:22:05.750+02:00 info vpxd[7FAAEC88F800] [Originator@6876 sub=Main] ** BEGIN SERVICES ** (time to active: 4668 ms)

2019-11-25T16:22:05.761+02:00 info vpxd[7FAADCDC7700] [Originator@6876 sub=Heartbeat opID=HeartbeatStartHandler-606df6c8] primary handler started

2019-11-25T16:22:05.787+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientInit-437266f5] Authenticated to LS old API for stubAdapter: [N5Vmomi19SoapStubAdapterImplE:0x00007faac830eea0]

2019-11-25T16:22:05.792+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseClient opID=LicenseClientInit-437266f5] VC license is already assigned

2019-11-25T16:22:05.792+02:00 info vpxd[7FAADD2D1700] [Originator@6876 sub=licenseClient opID=LicenseClientInit-437266f5] License client started

2019-11-25T16:22:05.871+02:00 error vpxd[7FAADDF6A700] [Originator@6876 sub=vmomi.soapStub[18]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2019-11-25T16:22:05.871+02:00 warning vpxd[7FAADDF6A700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:05.890+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Discovered license service endpoint URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.890+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.890+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using ls URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.950+02:00 error vpxd[7FAADC2B1700] [Originator@6876 sub=vmomi.soapStub[20]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2019-11-25T16:22:05.950+02:00 warning vpxd[7FAADC2B1700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Discovered license service endpoint URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceDiscovery opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using license service endpoint URL from cache: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseServiceProxy opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Using ls URL: https://vcenter.pash.gr:443/ls/sdk

2019-11-25T16:22:05.969+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=licenseAssignmentCleaner opID=LicenseClientProcessInventoryLoadedAsync-59c09d07] Run on timer every: 86400 seconds

2019-11-25T16:22:07.049+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=vpxLro opID=2dfc6514] [VpxLRO] -- BEGIN lro-7 -- SessionManager -- vim.SessionManager.loginByToken -- 5297b522-1f4b-91ef-9729-b4adce59f626

2019-11-25T16:22:07.052+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(vsphere.local\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4, false)

2019-11-25T16:22:07.097+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(vsphere.local\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4, false) res: VSPHERE.LOCAL\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4

2019-11-25T16:22:07.097+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=AuthorizeManager opID=2dfc6514] [Auth]: User VSPHERE.LOCAL\machine-8ed13e22-120c-4b1c-8975-08c637cad0f4

2019-11-25T16:22:07.097+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsAdministrators, true)

2019-11-25T16:22:07.115+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsAdministrators, true) res: VSPHERE.LOCAL\HmsAdministrators

2019-11-25T16:22:07.115+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsRemoteUsers, true)

2019-11-25T16:22:07.133+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\HmsRemoteUsers, true) res: VSPHERE.LOCAL\HmsRemoteUsers

2019-11-25T16:22:07.133+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\SRM Remote Users, true)

2019-11-25T16:22:07.151+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\SRM Remote Users, true) res: VSPHERE.LOCAL\SRM Remote Users

2019-11-25T16:22:07.151+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrators, true)

2019-11-25T16:22:07.169+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=[SSO] opID=2dfc6514] [UserDirectorySso] GetUserInfo(VSPHERE.LOCAL\Administrators, true) res: VSPHERE.LOCAL\Administrators

2019-11-25T16:22:07.173+02:00 info vpxd[7FAADD1CF700] [Originator@6876 sub=vpxLro opID=15501ba2] [VpxLRO] -- BEGIN lro-8 -- ServiceInstance -- vim.ServiceInstance.retrieveContent -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)

2019-11-25T16:22:07.174+02:00 info vpxd[7FAADD1CF700] [Originator@6876 sub=vpxLro opID=15501ba2] [VpxLRO] -- FINISH lro-8

2019-11-25T16:22:07.174+02:00 info vpxd[7FAAD7060700] [Originator@6876 sub=vpxLro opID=2dfc6514] [VpxLRO] -- FINISH lro-7

2019-11-25T16:22:07.181+02:00 info vpxd[7FAADC332700] [Originator@6876 sub=vpxLro opID=72e666b4] [VpxLRO] -- BEGIN lro-9 -- EventManager -- vim.event.EventManager.postEvent -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)

2019-11-25T16:22:07.182+02:00 info vpxd[7FAADC332700] [Originator@6876 sub=vpxLro opID=72e666b4] [VpxLRO] -- FINISH lro-9

2019-11-25T16:22:07.183+02:00 info vpxd[7FAADC230700] [Originator@6876 sub=vpxLro opID=7460ee5a] [VpxLRO] -- BEGIN lro-10 -- ServiceInstance -- vim.ServiceInstance.retrieveContent -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)

2019-11-25T16:22:07.185+02:00 info vpxd[7FAADCF4A700] [Originator@6876 sub=Heartbeat opID=HeartbeatModuleStart-5c8022d1] Certificate not available, starting hostsync for host: host-45

2019-11-25T16:22:07.186+02:00 info vpxd[7FAADC230700] [Originator@6876 sub=vpxLro opID=7460ee5a] [VpxLRO] -- FINISH lro-10

2019-11-25T16:22:07.188+02:00 info vpxd[7FAAD7CF9700] [Originator@6876 sub=vpxLro opID=409434ca] [VpxLRO] -- BEGIN lro-12 -- SessionManager -- vim.SessionManager.logout -- 5297b522-1f4b-91ef-9729-b4adce59f626(52d52411-29bc-5461-4648-3125e11adcc3)

2019-11-25T16:22:07.190+02:00 info vpxd[7FAAD7CF9700] [Originator@6876 sub=vpxLro opID=409434ca] [VpxLRO] -- FINISH lro-12

2019-11-25T16:22:07.200+02:00 info vpxd[7FAAD7A74700] [Originator@6876 sub=HostAccess opID=HostSync-host-45-7c09267-TaskLoop-73cbac2a] Using vpxapi.version.version11 to communicate with vpxa at host esxi02.pash.gr

2019-11-25T16:22:08.257+02:00 info vpxd[7FAADDF6A700] [Originator@6876 sub=vpxLro opID=lro-13-66926e8] [VpxLRO] -- BEGIN lro-13 --  -- VsanHostConfigIssuesLRO --

2019-11-25T16:22:08.257+02:00 info vpxd[7FAADDF6A700] [Originator@6876 sub=vpxLro opID=lro-13-66926e8] [VpxLRO] -- FINISH lro-13

2019-11-25T16:22:08.257+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [StringTable::Expand]: nBkts = 256

-->

2019-11-25T16:22:08.325+02:00 info vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [StringTable::Expand]: nBkts = 256

-->

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Execute result code: -1

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] SQL execution failed: INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Execution elapsed time: 1 ms

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Statement diagnostic data from driver is 23505:0:7:ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";

--> Error while executing the query

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Connection diagnostic data from driver is HY000:0:110:

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] Bind parameters:

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [0]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 311

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [1]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 4000

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [2]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 1

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [3]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 1

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [4]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 1

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [5]datatype: 11, size: 4, arraySize: 0

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "ok"

2019-11-25T16:22:08.386+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [6]datatype: 11, size: 20, arraySize: 0

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "VM Network"

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [7]datatype: 11, size: 34, arraySize: 0

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "Network adapter 1"

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [8]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 100

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [9]datatype: 1, size: 4, arraySize: 0

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = 7

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [10]datatype: 11, size: 52, arraySize: 0

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "vim.vm.device.VirtualE1000"

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] [11]datatype: 11, size: 2146, arraySize: 0

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] value = "<obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:vim25" versionId="6.5" xsi:type="VirtualE1000"><key>4000</key><deviceInfo><label>Netw..."

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [Vdb::IsRecoverableErrorCode] Unable to recover from 23505:7

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [Vdb::IsRecoverableErrorCode] Unable to recover from HY000:110

2019-11-25T16:22:08.387+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] [VdbStatement] SQLError was thrown: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";

--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"

2019-11-25T16:22:08.392+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=vpxCommon opID=HostSync-host-45-7c09267] [Vpxd_HandleVmRootError] Received unrecoverable VmRootError. Generating minidump ...

2019-11-25T16:22:08.392+02:00 error vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";

--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"

2019-11-25T16:22:08.395+02:00 panic vpxd[7FAADC73A700] [Originator@6876 sub=Default opID=HostSync-host-45-7c09267]

-->

--> Panic: Unrecoverable VmRootError. Panic!

--> Backtrace:

--> [backtrace begin] product: VMware VirtualCenter, version: 6.5.0, build: build-4602587, tag: vpxd, cpu: x86_64, os: linux, buildType: release

--> backtrace[00] libvmacore.so[0x002B9EA8]: Vmacore::System::Stacktrace::CaptureFullWork(unsigned int)

--> backtrace[01] libvmacore.so[0x001B4E90]: Vmacore::System::SystemFactoryImpl::CreateBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)

--> backtrace[02] libvmacore.so[0x002B7E5B]

--> backtrace[03] libvmacore.so[0x002B7F36]: Vmacore::PanicExit(char const*)

--> backtrace[04] vpxd[0x00722E78]

--> backtrace[05] vpxd[0x00738629]

--> backtrace[06] vpxd[0x0073A837]

--> backtrace[07] vpxd[0x0073FB68]

--> backtrace[08] libvmacore.so[0x0023ED6F]

--> backtrace[09] libvmacore.so[0x0023F056]

--> backtrace[10] libvmacore.so[0x002BF67B]

--> backtrace[11] libpthread.so.0[0x00007454]

--> backtrace[12] libc.so.6[0x000E8E2D]

--> [backtrace end]

[daphnissov · Accepted solution]

If you had searched prior to posting (as I advised you earlier), you would have found a thread like this: SOLVED: Could not connect to one or more vCenter Server systems:

[oBull]

Thanks and sorry, I'll try this and reply if fixed as correct answer.

[oBull]

Good morning,

Thanks for your time and suggestion, the noted article solved my problem and now my vpxd service is up and running after deleting duplicates in postgres VCDB.

Have a nice day