Solved: Can't sign into VCSA with AD credentials

7007VM7007 · ‎07-29-2017

I upgraded my vCenter Appliance from 6.5e to update 1 yesterday and upgraded the ESXi hosts from 6.5d to 6.5 update 1. There were no issues/errors during the upgrade.

Before performing the upgrade I have been logging into the Flash or HTML5 web client using my AD credentials. My VCSA is domain joined and only had an IPv4 address.

After the upgrade I added an IPv6 address to the VCSA and can no longer login to the VCSA using my AD credentials, I get the "Invalid credentials" error after clicking the Login button. This only started happening after adding a static IPv6 address to the VCSA. I did add a reverse DNS record for the VCSA FQDN. When I only had an IPv4 address SSO worked but now that I have dual IPv4/IPv6 addresses AD logins are failing and the only way for me to login is to use the administrator@domain.local account.

If I run the following on the VCSA while logged in as root and using the shell:

cat /var/log/messages | grep -i "error"

I get this:

2017-07-29T07:08:59.463241+00:00 localhost ntpd[24585]: kernel reports TIME_ERROR: 0x2041: Clock Unsynchronized

2017-07-29T07:26:38.000442+00:00 localhost lsassd[1129]: 0x7f898115c700:Failed to sync system time [error code: 40075]

2017-07-29T06:16:33.103697+00:00 localhost cli: root SSO initialization error: [Errno 111] Connection refused

This got me thinking about time...

So I checked my VCSA time and it was one hour off compared to the ESXi servers and domain but the VCSA does use an NTP server (which is my domain controller). I have set both IPv4/IPv6 addresses for the NTP server.

So I logged into https://vcsa.domain:5480/ and it was using the correct NTP settings and syncing but it was one hour off. The strange thing is, the time zone field is empty. I tried setting it a few times (and rebooting) but its still empty.

Not sure if the lack of a time zone is the cause of this issue but how can I get my SSO with AD working again?

I also tried disabling NTP and setting the time manually on the VCSA but it kept changing back to the incorrect time (even after a reboot).

All my SSO logins are currently broken and I can't do Citrix MCS as it keeps saying the username/password is incorrect so plenty has gone wrong.

Is this a time issue and if so how do I resolve it? Are there any other logs that would be helpful in troubleshooting this?

Added root SSO connection refused error from log

JordonC · ‎08-08-2017

I have been able to fix my Test environment by doing the following. My problem is the hostname was changed somehow after the update to U1. Looked correct everywhere except when checking using ssh. This is what I did to fix

1. First I logged into the Web Console with administrator@vsphere.local

2. Go to Administration and go to configuration and look at the certificates. Check if they are they for the correct FQDN

3. Next Go to hosts and cluster and right click on top VCenter Name and go to settings. Expand Runtime Settings. Check if the correct FQDN name is there also.

4. After all those look good SSH into the VCenter server and run the command hostname. If localhost.localdom comes up U1 probably defaulted to this hosts name. It needs to be changed back to what your certificate says.

5.

Connect to the vCenter Server Applaince with an SSH session and log in as root.
Run the command shell.set --enabled=True to enable the shell interface
Run shell to activate the bash shell.
Remove the vCenter Server Appliance from the domain by running this command:

/opt/likewise/bin/domainjoin-cli leave
Correct the hostname of the vCenter Server Applaince by running this command: !!Make sure host name matches your existing certificates!!

/opt/vmware/share/vami/vami_set_hostname vCenter-Appliance-FQDN

Example:

/opt/vmware/share/vami/vami_set_hostname vcenter01.company.com

After completing, join the Appliance to the domain using:

/opt/likewise/bin/domainjoin-cli join domain user

Example:

/opt/likewise/bin/domainjoin-cli join company.net Administrator
When prompted, enter the password for the domain user
Reboot the vCenter Server Appliance

This worked for me. You may have to remove the identity and re-add it but I did not. Once my host name matched the certificate I was able to log in again with ad credentials.

View solution in original post

7007VM7007 · ‎07-29-2017

So to try and troubleshoot this further I removed the vCenter appliance from the domain and then removed the identity source. Now I can't re-join the VCSA to the domain. It gives me this error each time:

The "Join active directory" operation failed for the entity with the following error message.

The format of the specified computer name is invalid [domain.com]

After a bit of research this is due to my VCSA having the name localhost.localdom which I find odd as I have always specified the name vcsa.domain.com for my appliance and have my DNS A/PTR records setup for vcsa.domian.com

So how can I rejoin my VCSA to the domain again? It looks like I can't rename the VCSA from what I have read and yet it was joined to the domain and working.

I'm at a loss with all this!

AhmedAtia · ‎07-29-2017

Hi,

I'm not sure if this would help, but does the FQDN contains illegal characters, such as underscore _ ?

We once had to spend many hours trying to troubleshoot a VMware Horizon environment for one of our customers, only to find that their AD domain had an "underscore"'; and according to the RFCs, this character shouldn't be used in DNS names.

About the time difference issue, I'm not sure why this happened, I'm going to deploy 6.5 U1 in a lab environment and see if that happens.

Thanks,

Atia

7007VM7007 · ‎07-29-2017

Hi Atia

No, my domain name (both NetBIOS and FQDN) only uses characters from the alphabet as follows: domain.com So no special characters, underscores, dashes etc.

Keen to hear your findings with 6.5 U1.

I just cannot get my VCSA joined to the domain and seem to be going around in circles.

7007VM7007 · ‎07-30-2017

UPDATE:

I ended up adding my Active Directory as an identity source using LDAP rather than using the integrated authentication option. This worked first time and I can now login to vCenter using my AD account. This is the good news.

The bad news is, I still can't join my VCSA to the domain (I know I don't have to now that I am using the LDAP option but that's not the point) and I also can't join my ESXi hosts to the domain for SSO.

So I'm not sure what's going on but joining either an ESXi host or a VCSA to the domain just doesn't work with 6.5 U1 for me.

JordonC · ‎07-31-2017

We are having the exact same issues after upgrading to 6.5 U1. If users use the Enhanced Authentication plugin they can get in by checking the box. But users on browsers that cant utilize the plugin can no longer log in. Typing username@domain.net or domain\username no longer works. I'm about to revert back because I'm getting constant calls.

JordonC · ‎07-31-2017

I reverted all production VCenter Servers back to 6.5.0.5500 and I'm able to authenticate again. Left our test environment on 6.5 U1 and AD auth is still broken on it. I'm getting ready to open a ticket with support. I don't think its a time issue because that server has the correct time zone and time. But I did experience the same issue as user 7007VM7007. 2 Out of the the 3 VCenter servers I could not change the time zone. It went from eastern time zone to blank zone after the update U1 and I could not change it back.

Jbir · ‎08-01-2017

I have the same problem and have also hit a blank wall. I have opened a support request with vmware as this must be a bug with U1.

Do you have smb1 disabled on your domain controllers? I had a similar issue in VCSA 6.0 when we disabled smb1 and couldn't authenticate using AD accounts, this was fixed in the 6.5 appliance as it has SMB 2 enabled but i can't help but think it is related. I have checked and SMB 2 is still enabled after the upgrade though.

jrwise · ‎08-01-2017

Having same issue. Reverted back to snapshot until issue is fixed. We have SMB1 disabled in our systems.

7007VM7007 · ‎08-01-2017

I also have SMB1 disabled on ALL my servers but as a test I enabled SMB1 on both my domain controllers and tried to join the VCSA to the domain again but that didn't work either.

Currently my VCSA and ESXi hosts are not in the domain anymore.

I am using Active Directory LDAP as my Identity Source currently and that is working fine but I would like to add my VCSA and ESXi servers back into the domain at some stage.

aavvalitorontoc · ‎08-01-2017

I have the same kind of problem too. I can't login with providing user credentials (like powercli or manually typing username and password) but using the passthrough authentication with plugin works. Have a ticket with vmware and awaiting call back.

JordonC · ‎08-01-2017

SMB doesn't seem to be the issue. On our test environment SMB1 is still enabled on the domain controllers and still have the same with with our test VCenter. I've opened a ticket and it's been over 24hrs without a response back. I would say it's currently a known issue without a fix

harmlessrobot · ‎08-01-2017

This broke for me after vsphere 6.5 update 1. Then I noticed in their release notes they said they do not support disjointed domain names. We have 3 dns names for machines and the appliance was not using the active directory name for the hostname but was using one of the others. I rebuilt the appliance using the ad hostname and logins work.

Jbir · ‎08-02-2017

We don't have disjointed domain names so that shouldn't be a problem and the release notes say that is fixed in U1, unless i'm reading it wrong.

I have noticed a new problem with deploying OVFs, although could have been there before as i haven't deployed one since upgrading to 6.5! When i select Deploy OVF Template i get the following error:

Does anyone else get this?

aavvalitorontoc · ‎08-02-2017

This fixed it for me

In disjoint domain namespace the domain users might fail to authenticate after you update to vSphere 6.5 Update 1

After you update a Platform Services Controller Appliance to vSphere 6.5 Update 1, in the disjoint domain namespace the users might fail to authenticate.

1. Log in to the Platform Services Controller Appliance as root and activate the bash shell.
2. Leave the domain by running the /opt/likewise/bin/domainjoin-cli leave command.
3. Reboot the appliance.
4. Delete the computer account on the Active Directory.
5. Log in to the appliance again and enable the bash shell.
6. Join to the domain by running the following command /opt/likewise/bin/domainjoin-cli join domain-name domain_admin_user
for example: /opt/likewise/bin/domainjoin-cli join vmware.com administrator
7. Reboot the appliance.

Remove from domain using command, reboot, wait 15 minutes, join to domain, reboot, wait 15 minutes, add authentication source using built-in authentication, reboot, and it works

7007VM7007 · ‎08-02-2017

When I try to run:

/opt/likewise/bin/domainjoin-cli join vmware.com administrator

I get this:

Error: Invalid hostname [code 0x000004ba]

The hostname may not be 'linux' or 'localhost'.

Running hostname gives me this:

localhost.localdom

Whats strange is, I have been really careful with using the FQDN to refer to my VCSA since it's installations which is: vcsa.domain.com

So I can't understand where the hostname "localhost.localdom" comes from.

BenBlock18 · ‎08-02-2017

I was having same issue not being able to login with AD credentials by typing it in, but the plug-in was working. I tried using the instructions to remove from the domain and re-add to the domain. This completed, but did not resolve the issue. My problem was the actual hostname was different then then the vCenter configured hostname. (I had tried to change it during the migration from windows to vcsa 6.5) In the appliance management page, under network, the host name was greyed out. I was able to change the hostname in the vsphere web client to match the actual hostname. After a reboot, I was able to login with my AD credentials.

7007VM7007 · ‎08-02-2017

I was able to change the hostname in the vsphere web client to match the actual hostname.

Would you mind providing the exact steps you did to achieve this because I couldn't find a way to change my hostname.

BenBlock18 · ‎08-02-2017

log into vsphere web client
go to hosts and clusters view
on left side, click on the top level (your vCenter server)
on right side click on Configure tab...Settings...General
on right side next to vCenter Server Settings, click on Edit... button.
in the pop-up windows, click on Runtime settings

From there you can edit vCenter Server name. After clicking OK, you'll need to reboot the VCSA.

7007VM7007 · ‎08-02-2017

log into vsphere web client
go to hosts and clusters view
on left side, click on the top level (your vCenter server)
on right side click on Configure tab...Settings...General
on right side next to vCenter Server Settings, click on Edit... button.
in the pop-up windows, click on Runtime settings
From there you can edit vCenter Server name. After clicking OK, you'll need to reboot the VCSA.

Thanks but I checked there and it has the correct FQDN and yet when I SSH into my VCSA and run "hostname" it shows localhost.localdom as my hostname.