Over the past few months (since upgrading to 5.5) we have started receiving LDAP error messages through email from vCD stating that there was an error when connecting to the LDAP server. I have pasted in below, the alert as well as output from both cell's logs. LDAP works fine but we will randomly get some of these errors from time to time. Does anyone know what might be happening or has run into this before?
Email Alert:
Error encountered connecting to LDAP server "FQDN.com" with username "domain\username". Error message was: "FQDN.com:port"
Cell01:
[root]# cat vcloud-container-debug.log | grep LDAP
[root]# cat vcloud-container-info.log | grep LDAP
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: com.vmware.ssdc.backend base.ldap.LdapSyncException: LDAP_SYNC_ERROR]
Caused by: com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
Cell02:
[root]# cat vcloud-container-info.log | grep LDAP
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
[root]# cat vcloud-container-debug.log | grep LDAP
2014-04-10 10:16:02,360 | DEBUG | pool-jetty-74 | JobManager | **** Doing operation JOB_LDAP_SYNC on ( com.vmware.vcloud.entity.ldap:893dc9ed-11fc-4815-862b-e68269476939) |
2014-04-10 10:16:02,360 | DEBUG | pool-jetty-74 | JobString | Job object - Object : (com.vmware.vclou d.entity.ldap:893dc9ed-11fc-4815-862b-e68269476939) operation name: JOB_LDAP_SYNC |
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
2014-04-10 10:16:14,076 | DEBUG | akimbitask-1 | JobString | Job object - Object : (com.vmware.vclou d.entity.ldap:893dc9ed-11fc-4815-862b-e68269476939) operation name: JOB_LDAP_SYNC | vcd=2235c059-5d47-4a97-bcac-751946f3148e,task=365f5d0 b-89b8-3212-80db-9fa702c0ec79
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
There isn't enough from what you posted to really see what was going on. Is there a 'caused by' line in the debug log level after the LdapSyncException? The one posted in generic for any Sync failure, but ideally there should be a subexception. What I see here is the ldap sync failed with a specific provider.
The Sync is just to update user information (first name, last name, etc). The LDAP login process is independent of syncing.
Is there any cadence to the error state/emails? Like a time of day, or day of week? Is the service account used for this LDAP connection used in other applications?
In respect to the upgrade to 5.5.x generation, the LDAP code didn't change but the JRE Version did (to a release of 1.7, when vCD 5.1.x was JRE 1.6).
I haven't noticed any specific cadence related to the errors since this one came from an org admin, who gets generic errors related to his org. I was able to find a little more in the log after doing a deeper search though:
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
at com.vmware.ssdc.backendbase.usermanagement.LdapSyncHelper.syncLdap(LdapSyncHelper.java:114)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl.syncLdap(LdapManagerImpl.java:333)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl.access$100(LdapManagerImpl.java:84)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl$3.run(LdapManagerImpl.java:370)
at com.vmware.ssdc.backendbase.CAkimbiTask._invokeChildUnsafe(CAkimbiTask.java:95)
at com.vmware.ssdc.backendbase.CAkimbiTask.access$500(CAkimbiTask.java:39)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.innerRun(CAkimbiTask.java:201)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.access$300(CAkimbiTask.java:120)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread$1.run(CAkimbiTask.java:139)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread$1.run(CAkimbiTask.java:128)
at com.vmware.vcloud.common.threadpool.ThreadContextExecutor.executeImpl(ThreadContextExecutor.java:69)
at com.vmware.vcloud.common.threadpool.ThreadContextExecutor.execute(ThreadContextExecutor.java:62)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.run(CAkimbiTask.java:153)
2014-04-10 10:16:14,073 | DEBUG | akimbitask-1 | CJob | updateFailedJob(com.vmware.ssdc.backendbase.ldap.LdapSyncException) with locale=en_US | vcd=2235c059-5d47-4a97-bcac-751946f3148e,task=365f5d0b-89b8-3212-80db-9fa702c0ec79
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
at com.vmware.ssdc.backendbase.usermanagement.LdapSyncHelper.syncLdap(LdapSyncHelper.java:114)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl.syncLdap(LdapManagerImpl.java:333)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl.access$100(LdapManagerImpl.java:84)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl$3.run(LdapManagerImpl.java:370)
at com.vmware.ssdc.backendbase.CAkimbiTask._invokeChildUnsafe(CAkimbiTask.java:95)
at com.vmware.ssdc.backendbase.CAkimbiTask.access$500(CAkimbiTask.java:39)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.innerRun(CAkimbiTask.java:201)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.access$300(CAkimbiTask.java:120)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread$1.run(CAkimbiTask.java:139)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread$1.run(CAkimbiTask.java:128)
at com.vmware.vcloud.common.threadpool.ThreadContextExecutor.executeImpl(ThreadContextExecutor.java:69)
at com.vmware.vcloud.common.threadpool.ThreadContextExecutor.execute(ThreadContextExecutor.java:62)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.run(CAkimbiTask.java:153)
2014-04-10 10:16:14,076 | DEBUG | akimbitask-1 | JobString | Job object - Object : (com.vmware.vcloud.entity.ldap:893dc9ed-11fc-4815-862b-e68269476939) operation name: JOB_LDAP_SYNC | vcd=2235c059-5d47-4a97-bcac-751946f3148e,task=365f5d0b-89b8-3212-80db-9fa702c0ec79
2014-04-10 10:16:14,109 | DEBUG | akimbitask-1 | CJob | No last pending job : [(com.vmware.vcloud.entity.ldap:893dc9ed-11fc-4815-862b-e68269476939)], status=[3] | vcd=2235c059-5d47-4a97-bcac-751946f3148e,task=365f5d0b-89b8-3212-80db-9fa702c0ec79
2014-04-10 10:16:14,112 | DEBUG | akimbitask-1 | CJob | Update last job : [(com.vmware.vcloud.entity.ldap:893dc9ed-11fc-4815-862b-e68269476939)], status=[3], [4/10/14 10:16 AM] | vcd=2235c059-5d47-4a97-bcac-751946f3148e,task=365f5d0b-89b8-3212-80db-9fa702c0ec79
2014-04-10 10:16:14,114 | ERROR | akimbitask-1 | CAkimbiTask | Exception thrown in Job | vcd=2235c059-5d47-4a97-bcac-751946f3148e,task=365f5d0b-89b8-3212-80db-9fa702c0ec79
com.vmware.ssdc.backendbase.ldap.LdapSyncException: LDAP_SYNC_ERROR
at com.vmware.ssdc.backendbase.usermanagement.LdapSyncHelper.syncLdap(LdapSyncHelper.java:114)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl.syncLdap(LdapManagerImpl.java:333)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl.access$100(LdapManagerImpl.java:84)
at com.vmware.ssdc.backendbase.ldap.LdapManagerImpl$3.run(LdapManagerImpl.java:370)
at com.vmware.ssdc.backendbase.CAkimbiTask._invokeChildUnsafe(CAkimbiTask.java:95)
at com.vmware.ssdc.backendbase.CAkimbiTask.access$500(CAkimbiTask.java:39)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.innerRun(CAkimbiTask.java:201)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.access$300(CAkimbiTask.java:120)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread$1.run(CAkimbiTask.java:139)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread$1.run(CAkimbiTask.java:128)
at com.vmware.vcloud.common.threadpool.ThreadContextExecutor.executeImpl(ThreadContextExecutor.java:69)
at com.vmware.vcloud.common.threadpool.ThreadContextExecutor.execute(ThreadContextExecutor.java:62)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.run(CAkimbiTask.java:153)
Thaaaaaaaaaaaaaaaaat ... didn't help. Is there any server side error to correspond? I forget if you are connecting to MS LDS or something else.
The source is a corporate AD server which we don't have access to so I was trying to rule out issues on the vCD side. I am able to do a manual sync without issue, but I think we may still get that email from time to time.
Bump...
Am experiencing this same error now with our vCD infrastructure and corporate Microsoft Active Directory. Any idea what resolved it?
javax.naming.CommunicationException: mgsops.net:389 [Root exception is java.net.ConnectException: Connection timed out]
- [mydomainname].net:389
- Connection timed out
com.vmware.vcloud.common.threadpool.ThreadContextExecutor.execute(ThreadContextExecutor.java:62)
at com.vmware.ssdc.backendbase.CAkimbiTask$InvokeChildThread.run(CAkimbiTask.java:153)
Caused by: javax.naming.CommunicationException: mgsops.net:389 [Root exception is java.net.ConnectException: Connection timed out]
TIA
Darren
Two notes to kick off:
1. You forgot to redact the hostname.net in two other places.
2. Please start a new thread. Reusing old threads usually causes confusion as to what the problem is, and there is no way for you to mark something as correct (since you didn't create the thread).
This isn't even a vCloud Director error. This is Java stating there is a timeout when connecting your domain. Can you try putting a specific domain controller into vCloud Director's configuration? Hopefully one on the same network as the Cells, to avoid firewalls or WAN latency.