VMware Cloud Community
Poestges
Contributor
Contributor

An error occurred while communicating with the remote host

Hi community,

I have a problem with my VC2.5 and ESX3.5.

50% of all actions results in "An error occurred while communicating with the remote host". The 2nd try mostly works, sometimes i have to do another try.

I start an action, the host disconnects and a few seconds later the host is back and i can give it another try. The error occures on every host i have.

Can anyone help me, please?

TIA,

Sascha

Reply
0 Kudos
48 Replies
jrenton
Hot Shot
Hot Shot

Could be a DNS issue. Do you have a DNS server in the DNS domain your ESX servers reside in? I had a similar issue and I added both hosts in my cluster to /etc/hosts file on each ESX server. Problem was that each ESX server in the cluster could not communicate with the other host.

Hope this helps.

John

Reply
0 Kudos
Poestges
Contributor
Contributor

Hi John,

the DNS was my first thought, too. I checked the hosts files on all ESX and VC. Everything is fine.

Sascha

Reply
0 Kudos
kjb007
Immortal
Immortal

Make sure that your hostname matches what you have configured. Meaning, make sure when you run the commands 'hostname' and 'hostname -s', that you see what you are supposed to see. Also, make sure you add the hosts using their FQDN, and not their IPs.

-KjB

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
Reply
0 Kudos
admin
Immortal
Immortal

Has this always been a problem?

Did you replace the VC ssl certificates with new ones?

Is the ESX server swapping heavily or low on disk space?

free -m

df -h

Have a look at the vpxa and hostd logs on the ESX server at the time of the disconnects for any errors/warnings:

/var/log/vmware/vpx/vpxa.log

/var/log/vmware/hostd.log

Reply
0 Kudos
Poestges
Contributor
Contributor

The hostnames are all ok.

The VI was running fine for 18 month, the strange behavior is for 1 month now.

I just received the error during a VM migration.

- The migration starts

- I got the error message,

- The host got disconnected for a few seconds and reconnects

- I checked the VM. It was migrated succesfully.

So It seems to me that the connections between the hosts are runing fine, and that the problem is between the VirtualCenter and the hosts.

Reply
0 Kudos
admin
Immortal
Immortal

So this happens with all your hosts and only for migration tasks?

Have you replaced the VirtualCenter ssl certs with new ones?

Have you tried restarting the VirtualCenter server service or re-creating the cluster?

Do they VC logs show anything unusual at the times of the disconnects?

Trigger the error, wait a few mins, then attach the latest vpxd log to a post. The logs are normally stored somewhere like C:\Documents and Settings\All Users\Application Data\VMware\VirtualCenter\Logs\

Reply
0 Kudos
Poestges
Contributor
Contributor

- This happens to all my hosts. I added 3 new hosts last week and they show the same behavior.

Not only for migration tasks, every tasks (like working with Snapshots, add Ports to a vSwitch, edit VM settings, etc.) can result to the error.

Sometimes I can work for hours and everything is allright, and sometimes nearly every action leads to the error.

- The VirtualCenter certs are still the original

- I restarted the service and server several times - no luck

I wait for the next error now, and I'll post the log.

Reply
0 Kudos
Poestges
Contributor
Contributor

I started a HBA rescan and the error occurred .. here is the log:

2008-06-24 13:56:28.098 'App' 1444 info] -- FINISH task-internal-122423 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122424 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- FINISH task-internal-122424 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- BEGIN task-7339 -- host-2517 -- vim.host.StorageSystem.refresh -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122425 -- host-8 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx1.nospam.com (x.x.128.181)

Retrieved host update to 1039

Completed host synchronization

-- FINISH task-internal-122425 -- host-8 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122426 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

Retrieved host update to 481

Completed host synchronization

-- FINISH task-internal-122426 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- FINISH task-7339 -- host-2517 -- vim.host.StorageSystem.refresh -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122427 -- -- vim.ServiceInstance.retrieveContent -- 2DFA95CB-B3C1-4407-8281-6A43212F5DA2

-- FINISH task-internal-122427 -- -- vim.ServiceInstance.retrieveContent -- 2DFA95CB-B3C1-4407-8281-6A43212F5DA2

-- BEGIN task-internal-122428 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

Retrieved host update to 483

Completed host synchronization

-- FINISH task-internal-122428 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122429 -- -- vmodl.query.PropertyCollector.retrieveContents -- 481804DF-8C81-4E0A-A8E5-54CDEFCDF902

-- FINISH task-internal-122429 -- -- vmodl.query.PropertyCollector.retrieveContents -- 481804DF-8C81-4E0A-A8E5-54CDEFCDF902

ConfigInfo saved in db

-- BEGIN task-internal-122430 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122430 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122431 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122431 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122432 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122432 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122433 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122433 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122434 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122434 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122435 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122435 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122436 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122436 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122437 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122437 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122438 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122438 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122439 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122439 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122440 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122440 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-7340 -- host-2517 -- vim.host.StorageSystem.rescanAllHba -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122441 -- host-487 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx5.nospam.com (x.x.128.192)

Retrieved host update to 711

Completed host synchronization

-- FINISH task-internal-122441 -- host-487 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122442 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

Retrieved host update to 483

Completed host synchronization

Updating vpxa for esx7.nospam.com (spec. genno 694)

-- FINISH task-internal-122442 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122443 -- host-476 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx6.nospam.com (x.x.128.193)

Retrieved host update to 497

Completed host synchronization

-- FINISH task-internal-122443 -- host-476 -- VpxdInvtHostSyncHostLRO.Synchronize

VpxdClientAdapter::InvokeOnSoap: (esx7.nospam.com, vim.host.StorageSystem.rescanAllHba) took 5546 ms

-- BEGIN task-internal-122444 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

Got vmacore exception: SSL Exception: error:00000001:lib(0):func(0):reason(1)

Backtrace:

backtrace[00] eip 0x0122b526

backtrace[01] eip 0x01176caa

backtrace[02] eip 0x0115192e

backtrace[03] eip 0x01141b38

backtrace[04] eip 0x0117e254

backtrace[05] eip 0x011d7e1f

backtrace[06] eip 0x011d7f3b

backtrace[07] eip 0x011d8364

backtrace[08] eip 0x011d55c6

backtrace[09] eip 0x0118ce06

backtrace[10] eip 0x0115bbff

backtrace[11] eip 0x013a618a

backtrace[12] eip 0x013a6332

backtrace[13] eip 0x013a81ff

backtrace[14] eip 0x013a9051

backtrace[15] eip 0x00510d11 (no symbol)

backtrace[16] eip 0x0051169f (no symbol)

backtrace[17] eip 0x00511af1 (no symbol)

backtrace[18] eip 0x013779ce

backtrace[19] eip 0x008630a2 (no symbol)

backtrace[20] eip 0x0053b0e5 (no symbol)

backtrace[21] eip 0x0053bad8 (no symbol)

backtrace[22] eip 0x006d5df6 (no symbol)

backtrace[23] eip 0x006dad6e (no symbol)

backtrace[24] eip 0x006e0e04 (no symbol)

backtrace[25] eip 0x00a1c06a (no symbol)

backtrace[26] eip 0x7c824829 GetModuleHandleA

Enter DAS_PROFILE UpdateDasStatus

Exit DAS_PROFILE UpdateDasStatus (0 ms)

-- FINISH task-internal-122444 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Awaited sync failed for esx7.nospam.com

-- FINISH task-7340 -- host-2517 -- vim.host.StorageSystem.rescanAllHba -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- ERROR task-7340 -- host-2517 -- vim.host.StorageSystem.rescanAllHba: vmodl.fault.HostCommunication:

(vmodl.fault.HostCommunication) {

dynamicType = <unset>,

msg = ""

}

-- BEGIN task-internal-122445 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- FINISH task-internal-122445 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- BEGIN task-internal-122446 -- -- vmodl.query.PropertyCollector.createFilter -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122446 -- -- vmodl.query.PropertyCollector.createFilter -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122447 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122447 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122448 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122448 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122449 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122449 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122450 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122450 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- ERROR task-internal-122450 -- -- vim.LicenseManager.queryUsage: vmodl.fault.InvalidType:

(vmodl.fault.InvalidType) {

dynamicType = <unset>,

argument = "Non-optional data field is unset: source, vim.LicenseManager.LicenseUsageInfo",

msg = ""

}

-- BEGIN task-internal-122451 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122451 -- -- vmodl.query.PropertyCollector.Filter.destroy -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122452 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122452 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- ERROR task-internal-122452 -- -- vim.LicenseManager.queryUsage: vmodl.fault.InvalidType:

(vmodl.fault.InvalidType) {

dynamicType = <unset>,

argument = "Non-optional data field is unset: source, vim.LicenseManager.LicenseUsageInfo",

msg = ""

}

-- BEGIN task-internal-122453 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

-- BEGIN task-internal-122454 -- -- vmodl.query.PropertyCollector.retrieveContents -- 481804DF-8C81-4E0A-A8E5-54CDEFCDF902

-- FINISH task-internal-122454 -- -- vmodl.query.PropertyCollector.retrieveContents -- 481804DF-8C81-4E0A-A8E5-54CDEFCDF902

Retrieved host update to 489

Enter DAS_PROFILE UpdateDasStatus

Exit DAS_PROFILE UpdateDasStatus (15 ms)

VM vim.VirtualMachine:vm-98: config changed, invalidating cache...

VM vim.VirtualMachine:vm-179: config changed, invalidating cache...

VM vim.VirtualMachine:vm-1839: config changed, invalidating cache...

VM vim.VirtualMachine:vm-167: config changed, invalidating cache...

VM vim.VirtualMachine:vm-3296: config changed, invalidating cache...

VM vim.VirtualMachine:vm-132: config changed, invalidating cache...

VM vim.VirtualMachine:vm-2576: config changed, invalidating cache...

-- BEGIN task-internal-122455 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

VM vim.VirtualMachine:vm-3251: config changed, invalidating cache...

-- FINISH task-internal-122455 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- BEGIN task-internal-122456 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122456 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- ERROR task-internal-122456 -- -- vim.LicenseManager.queryUsage: vmodl.fault.InvalidType:

(vmodl.fault.InvalidType) {

dynamicType = <unset>,

argument = "Non-optional data field is unset: source, vim.LicenseManager.LicenseUsageInfo",

msg = ""

}

Completed host synchronization

Updating vpxa for esx7.nospam.com (spec. genno 694)

Enter DAS_PROFILE UpdateDasStatus

Exit DAS_PROFILE UpdateDasStatus (0 ms)

InvtHostSyncLRO::StartWork took 4078 ms

-- FINISH task-internal-122453 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122457 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122457 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122458 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- FINISH task-internal-122458 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- BEGIN task-internal-122459 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122459 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122460 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122460 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122461 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122461 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122462 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122462 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122463 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122463 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122464 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122465 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122465 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122466 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122466 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

ConfigInfo saved in db

-- FINISH task-internal-122464 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122467 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122467 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

ConfigInfo saved in db

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 childEntity.length

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 childType

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 disabledMethod

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 effectiveRole

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 name

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 overallStatus

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 parent

-- BEGIN task-internal-122468 -- -- vim.option.OptionManager.queryView -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- FINISH task-internal-122468 -- -- vim.option.OptionManager.queryView -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- BEGIN task-internal-122469 -- vm-132 -- vim.VirtualMachine.acquireMksTicket -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

-- FINISH task-internal-122469 -- vm-132 -- vim.VirtualMachine.acquireMksTicket -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

-- BEGIN task-internal-122470 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 childEntity.length

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 childType

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 disabledMethod

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 effectiveRole

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 name

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 overallStatus

ERProviderImpl<BaseT>::_GetChanges: Aggregate version Overflow group-d1 parent

-- FINISH task-internal-122470 -- -- vim.LicenseManager.queryUsage -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122471 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

Retrieved host update to 493

-- BEGIN task-internal-122472 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

Completed host synchronization

-- FINISH task-internal-122472 -- -- vmodl.query.PropertyCollector.retrieveContents -- 4A939008-5249-45C5-ACCA-BB0B02E93E9D

Updating vpxa for esx7.nospam.com (spec. genno 696)

-- FINISH task-internal-122471 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

ConfigInfo saved in db

-- BEGIN task-internal-122473 -- host-8 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx1.nospam.com (x.x.128.181)

Retrieved host update to 1040

Completed host synchronization

-- FINISH task-internal-122473 -- host-8 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-7341 -- host-2517 -- vim.host.StorageSystem.rescanVmfs -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

-- BEGIN task-internal-122474 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

SSLVerifyCertAgainstSystemStore: The remote host certificate has these problems:

  • A certificate in the host's chain is based on an untrusted root.

SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed despite the error

Retrieved host update to 497

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

Completed host synchronization

Updating vpxa for esx7.nospam.com (spec. genno 698)

-- FINISH task-internal-122474 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- FINISH task-7341 -- host-2517 -- vim.host.StorageSystem.rescanVmfs -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- BEGIN task-internal-122475 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

-- FINISH task-internal-122475 -- -- vmodl.query.PropertyCollector.retrieveContents -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

ConfigInfo saved in db

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

PropertyJournalImpl::HasChanged: Overflow summary.accessible

PropertyJournalImpl::HasChanged: Overflow summary.name

PropertyJournalImpl::GetChangeMap: Overflow , summary.accessible

PropertyJournalImpl::GetChangeMap: Overflow , summary.name

-- BEGIN task-internal-122476 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

Synchronizing host: esx7.nospam.com (x.x.128.171)

Retrieved host update to 497

Completed host synchronization

Updating vpxa for esx7.nospam.com (spec. genno 700)

-- FINISH task-internal-122476 -- host-2517 -- VpxdInvtHostSyncHostLRO.Synchronize

-- BEGIN task-internal-122477 -- -- vmodl.query.PropertyCollector.retrieveContents -- 481804DF-8C81-4E0A-A8E5-54CDEFCDF902

-- FINISH task-internal-122477 -- -- vmodl.query.PropertyCollector.retrieveContents -- 481804DF-8C81-4E0A-A8E5-54CDEFCDF902

Reply
0 Kudos
admin
Immortal
Immortal

Okay so here's a summary of the relevant errors I found:

2008-06-24 13:56:51.411 'App' 2088 error VpxdVmomi Got vmacore exception: SSL Exception: error:00000001:lib(0):func(0):reason(1)

2008-06-24 13:56:51.426 'App' 2088 error VpxdVmomi Backtrace:

backtrace00 eip 0x0122b526 mailto:?GenerateCoreDump@System

....

2008-06-24 13:56:51.723 'App' 2748 warning VpxdInvtHost Awaited sync failed for esx7.nospam.com
2008-06-24 13:56:51.723 'App' 2748 info VpxLRO -- FINISH task-7340 -- host-2517 -- vim.host.StorageSystem.rescanAllHba -- 44D70674-39D3-4A4D-A25A-C4F27E6DC206
2008-06-24 13:56:51.723 'App' 2748 info VpxLRO -- ERROR task-7340 -- host-2517 -- vim.host.StorageSystem.rescanAllHba: vmodl.fault.HostCommunication:
(vmodl.fault.HostCommunication) {
dynamicType = , 
argument = "Non-optional data field is unset: source, vim.LicenseManager.LicenseUsageInfo", 
msg = ""
}

......

PropertyJournalImpl::HasChanged: Overflow

Reply
0 Kudos
admin
Immortal
Immortal

It appears to be an SSL cert issue. Are you sure no-one changed the SSL certs on either VC or the ESX servers?

Try to re-create the ESX SSL certs:

1. If enabled, disable Virtual Machine Auto-start through the VI Client by clicking the host --> Configuration --> Virtual Machine Startup/Shutdown --> Properties

2. Disconnect and remove the host from VC by right-clicking it in the VI Client and choosing Disconnect then Remove

3. Login to the ESX server and stop hostd: service mgmt-vmware stop

4. Backup the old certs:

mkdir /root/ssl-old

cp /etc/vmware/ssl/* /root/ssl-old

5. Delete the certs: rm -rf /etc/vmware/ssl/*

6. Start hostd: service mgmt-vmware start

7. Reconnect the host to VC through the VI Client

8. Re-enable Virtual Machine auto-start if it was enabled.

Reply
0 Kudos
kjb007
Immortal
Immortal

Have you looked at our licensing? There are errors related to that in your log below.

I would also open an SR with vmware.

-KjB

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
Reply
0 Kudos
Poestges
Contributor
Contributor

@appk: tried it, but no luck Smiley Sad

@kjb: licences are fine, don't know what this error means

i think i have to start a SR with VMware, too

Thanks so far,

Sascha

Reply
0 Kudos
admin
Immortal
Immortal

Try to copy over the rui.crt file from the ESX server to the VC server and install it by right-clicking it and clicking Install.

Restart the VC service after its installed.

Reply
0 Kudos
admin
Immortal
Immortal

Have you made any progress on this?

Did you install the ESX cert on the VC server?

Did VMware support find anything?

Reply
0 Kudos
Poestges
Contributor
Contributor

i copied and installed the certs to server, but still no luck Smiley Sad

was waiting for results overnight, so i haven't contacted VMware support yet. I'm going to do so now.

Reply
0 Kudos
admin
Immortal
Immortal

It's a strange one.

I'm wondering if the certs have expired - are both the VC and ESX cert expiry dates okay?

Do you happen to have another set of the openssl libraries installed?

Do a search for ssleay32.dll and libeay32.dll. They should be in the VC install directory.

Have you changed the security policy of the server regarding certs/encryption?

Have you installed any other applications or service packs before it broke?

Reply
0 Kudos
Poestges
Contributor
Contributor

The expiry dates are oaky.

I haven't changed any policies or installed any other applications on the VC.

ssleay32.dll and libeay32.dll are in the install folder.

The strange thing is, that the error appeares sporadicly.

Reply
0 Kudos
kjb007
Immortal
Immortal

I would open an SR with vmware. Are you up-to-date on patches?

-KjB

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
Reply
0 Kudos
Poestges
Contributor
Contributor

I have opened a SR today. Patches are up-to-date

Reply
0 Kudos