Need help troubleshooting LDAPS configuration on vCenter 8

[pricemc1]

I am trying to configure an identity source in vCenter 8 using LDAPS with Active Directory on a new vCenter 8 implementation. I have been able to do this successfully many times in the past on vCenter 7 instances so I'm familiar with the procedure and requirements. This is my first time doing it though on a vCenter 8 VCSA, but nothing really looks different than how it was done in earlier versions.  Unfortunately, I seem to be running into some issues when I try to add the identity source using LDAPS. Adding it using standard LDAP works fine though.  I am receiving the following error when trying to configure the identity source for LDAPs though:

"Cannot configure identity source due to Failed to probe provider connectivity [URI: ldaps://domainl ]; tenantName [vsphere.local], userName [User] Caused by: Can’t contact LDAP server."

For troubleshooting I have done the following:

• I was able to contact and query the LDAP servers (Windows Server 2022 DCs) using 'ldp.exe' to test LDAPS from Windows clients that I tested from.  I was able to bind and authenticate using the domain account credentials I specified in the identity source configuration.
• I was able to confirm that the LDAPS servers are presenting the correct certificate by using Openssl to display the certificates being presented on port 636/3269.
• I was also able to verify network connectivity and proper name resolution from the VCSA to the LDAPs servers from the VCSA CLI using 'ping', 'dig', and 'nslookup'. Both forward and reverse lookups seem to be fine from the VCSA and Windows servers. All of these systems are on the same network. There is no router or firewall between any of them (other than OS level firewalls built into Windows and VCSA).
• When I ran 'ldapsearch' command from the VCSA's CLI it returns the no errors that indicate LDAPS is not working when being queried.
• I confirmed that the CA certificates for the CAs that issued the LDAPS certificates to the Windows servers have been added to the trusted list in vCenter.

I'm thinking my next troubleshooting step should be to review any relevant VCSA log files that would have detailed information on what is going on when I try to add the LDAPS identity source. I'm not sure what log files I should be checking though. If anyone can point me to the correct logs or has some additional troubleshooting advice it would be greatly appreciated.

Thanks in advance,

Mike

[pricemc1]

I checked the vmware-identity-sts.log log file and there is nothing really showing any errors or anything suspicious. I can't really even find any reference to the ldaps service account I'm using for testing the LDAPS identity source except for when its being used for regular LDAP instead of when I try to use it for testing LDAPS. If you really want I can attach the log file but I don't think there's anything to be found in that one.

I'm not much of a netcat person. Could you give me the exact nc command syntax you would like me to run and I'll run that on the VCSA?

If your looking to verify the port is open I'm pretty sure at some point I ran a curl -v from the VCSA to confirm I could reach 636/3269 from the VCSA to the AD LDAPS host. 

Thanks for your assistance...

[SureshKumarMuth]

If you have already checked using curl -v then thats fine. Since we are running out of options, I can suggest one thing just to isolate the issue. Check if you can quickly deploy a VCSA 7.x and try to add identity source with same settings. This would help us to conclude if this is due to the version changes ... may be based on the result we can find next course of action. You may also raise a vmware support ticket where they can deep dive into logs.

[pricemc1]

I will give that a shot.  Before I try that I think I will also try issuing another server LDAP cert with a shorter expirationation date and testing with that as well. Unfortunately it will take me a few days to get either of these done due to other life activities so if you think of anything else in between please let me know. Once I have those done I'll post my results.

[Ajay1988]

Share the inputs provided on identity source .

From vCSA: share the output. 

openssl s_client -connect <ad-domain>:636 -showcerts

Test the ports from vCSA

VCSA to AD ports : IP is for AD 

nc -zv -w 5 <IP> 88

nc -zv -w 5 <IP> 135

nc -zv -w 5 <IP> 389

nc -zv -w 5 <IP> 464

nc -zv -w 5 <IP> 3268

nc -zv -w 5 <IP> 3269

[pricemc1]

Here are the netcat command outputs:

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 88
AD01DC01.ad1.lab [10.0.1.11] 88 (kerberos) open

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 135
AD01DC01.ad1.lab [10.0.1.11] 135 (epmap) open

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 464
AD01DC01.ad1.lab [10.0.1.11] 464 (kpasswd) open

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 3268
AD01DC01.ad1.lab [10.0.1.11] 3268 (msft-gc) open

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 3269
AD01DC01.ad1.lab [10.0.1.11] 3269 (msft-gc-ssl) open

[pricemc1]

Attached is a screen shot of the identity source config.

[pricemc1]

Attached are the results from the openssl command.

[pricemc1]

Here's a couple others I may have left out.

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 389
AD01DC01.ad1.lab [10.0.1.11] 389 (ldap) open

root@vcenter0 [ ~ ]# nc -zv -w 5 ad01dc01.ad1.lab 636
AD01DC01.ad1.lab [10.0.1.11] 636 (ldaps) open

[Ajay1988 · Accepted solution]

Okay..So no issues with ports . Seems I missed checking the first update ....

So you are able to add with ldap. only ldaps is giving issues . 
This is surely with the certs.
Can you get the certs duration for less validity ? Keep it expiring before 2049 and check.

I recall we had an issue with more than 20 yrs cert in 6.5 and that was increased .  

[pricemc1]

Sorry, missed this reply for some reason. I will try this out in the next 24-48 hours and reply with the results. I wouldn't be surprised if that is the issue though. 

[pricemc1]

AJ,

You appear to have found the issue. I issued a new certificate with a 10 year expiration length and it worked just fine. Seems like whatever issue they had earlier with cert expiration lengths has either come back in vSphere 8 or was never truly fixed earlier. Perhaps being a VMware employee, you can file a bug report for this?

Thanks very much for your help...

Mike

[Ajay1988]

Great. Doesn't qualify for a bug....I have to check what they increased it to from 20 yrs. Your certs are 30 and 35 yrs. 
Which means we need to limit ldaps certs under the said limits. 
I will try to get this KB updated >> https://kb.vmware.com/s/article/2041378

[pricemc1]

Well if its not a bug then it definitely needs to be documented... I actually think the issue should have its own KB but it might be nice to have it mentioned in the one you cited as well. Is there a reason you wouldn't consider it a bug? Is there something in the LDAPS RFC or other PKI  standard that prohibits a certificate expiration length beyond 20 years? (granted, I know its bad practice, but it's a lab). If there isn't something like that then I would consider it a bug. The longer expiration certs had worked with everything else in my lab (AD, other apps, etc...) with no issues so that seems to point to this being something specific to vCenter, implying a bug...

[Ajay1988]

KB definitely needs an update and documented. I will try to reach the contacts to get it updated. 
Well in short for me a bug is something which doesn't work the way it is coded or developed... (Could be a product limitation) .
Here until 6.5 version some update they had the ldaps cert support only for 20 yrs and increased it later. But I don't recall what is was increased to.. ( I think may be less than 25 or 30 yrs). 

[rrivera120]

This is the way:

https://www.virten.net/2023/07/enabling-active-directory-ldap-ldaps-authentication-in-vcenter-8-0/