VMware Cloud Community
WhiskyTangoFoxt
Enthusiast
Enthusiast
Jump to solution

Unable to join VCSA 6 u1 back to domain. Error messages are not found anywhere online.

I wondering if someone can help me troubleshoot SSO with a VCSA running 6u1. PLEEASE!!  This was upgraded about a week ago from 6.0 and had no problems until it decided to fall apart on Monday. Around that time we had issues with our DCs, I'm unsure if it was related.

On Monday Veeam backups using a domain account to access VCenter stopped working. Authentication error. Trying to sign into VCenter Web client with SSO which weI have been doing for years also failed. Signing in as root to web client failed as well.

Signing into the C++ client worked for all logins with no problems, but really limited what we can do. So I unjoined the domain, and attempted to rejoin without success.

I can sign into the web client with administrator@vsphere.local, trying to join the domain as we have done before results in "Idm client exception: Error trying to join AD, error code [31], user..." - no reference to an error 31 anywhere. Yes the username is formatted as administrator@domain.net. The computer account has been recreated on the domain. Connectivity to the DC is fine, because if I put the password in wrong, it tells me the authentication has failed. All services on the VCSA are started except for the Auto-deploy service.

I've tried through SSH logged in as root (it tells me that administrator@vsphere.local has no shell access) /opt/likewise/bin/domainjoin-cli join domain.net administrator@domain.net results in ERROR_GEN_FAILURE [code 0x0000001f], again an error that has any results that are relevant.

I'm unable to create log bundles either through the web client or through the C++ client, I suspect due to a space issue on the VCSA which I haven't been able to resolve. Running VSAN performance monitoring seem to chew up space until I turned it off, but haven't found any resources online as to where to find them or how to remove them.

I have a snapshot that was done before I updated 8 days ago to 6u1, but I'm afraid that everything I've done since then will fall apart.This VCenter is managing a production VSAN, and I can't have it go down.

Can someone please help me sort this mess out? With the lack informative error messages I'm not sure where to start!

Thanks,

B

1 Solution

Accepted Solutions
WhiskyTangoFoxt
Enthusiast
Enthusiast
Jump to solution

Woohoo! I finally figured it out.

To start with I set up a new VCenter server as a trial, nothing configured, just deployed from the 6u1 iso. I found that it wasn't automatically joined to the domain and attempted to do so. SAME ERROR! So the problem isn't with VSphere. In the past few weeks we had introduced two new 2012 R2 DCs and retired one of our 2008 R2 DCs. I shut down the 2012 DCs and tried again with only the old DC on the network. It worked!!!! I was able to join the test VCSA to the domain, and after a reboot saw the leave domain button available.

So I disjoined it and then attempted to rejoin the domain while running wireshark on the NIC of the 2008R2 DC. Then I brought up a 2012R2 DC, waited 10 minutes or so then shut down the 2008R2. Again I ran wireshark but this time on the NIC of the 2012R2 server. I compare the results of a successful and unsuccessful join and saw the 2012R2 DC had a lack of SMB packets outbound to the VCSA.The server service was running on the 2012 DC, but Microsoft's recommendation was to change the srv.sys startup from auto to manual on server 2012 R2. I thought that this was odd at the time, but we changed it when we set up the new DCs to satisfy the BPA. I compared it to the registry setting on the 2008R2 DC. which was set to auto start.

So to allow VCSA to join the domain when you are getting the 31 error, undo this

http://social.technet.microsoft.com/wiki/contents/articles/21104.srv-sys-should-be-set-to-start-on-d...

on your server 2012R2 DCs if it has been implemented as it causes the domain join to fail.

Command prompt: sc config srv start=auto

or

Registry: HKLM:\System\CurrentControlSet\Services\srv\ from 3 to 2


Hope this saves someone else the amount of time it took me to figure it out! I guess I can cancel my support request now...


View solution in original post

11 Replies
WhiskyTangoFoxt
Enthusiast
Enthusiast
Jump to solution

Furthermore, I was able to back up the current VC database to my local machine, then restored VCenter from when it was last working to another host in the cluster. After checking that the MAC is the same, and restoring the current VC database I rebooted.

Again I find it disjoined from the domain, and I am still unable to rejoin it. So now I'm really stuck.

Backup and restore do nothing for broken SSO, nor does removing and re-joining. Is the SSO config part of the embedded database? I've looked at "Location of vCenter Single Sign-On log files for vCenter Server 5.1 and 5.5 (2033430)" but the log info looks irrelevant as far as V6 is concerned.

Demoting an old DC seems to be what has done it in, and although there are plenty of articles out there on the DC demotion breaking SSO, they all reference an earlier version of VCenter or VC running on Windows.

Space issue is cleared up, I'm able to generate log bundles, but don't see anything helpful in the SSO logs.

Anyone?

Reply
0 Kudos
WhiskyTangoFoxt
Enthusiast
Enthusiast
Jump to solution

Woohoo! I finally figured it out.

To start with I set up a new VCenter server as a trial, nothing configured, just deployed from the 6u1 iso. I found that it wasn't automatically joined to the domain and attempted to do so. SAME ERROR! So the problem isn't with VSphere. In the past few weeks we had introduced two new 2012 R2 DCs and retired one of our 2008 R2 DCs. I shut down the 2012 DCs and tried again with only the old DC on the network. It worked!!!! I was able to join the test VCSA to the domain, and after a reboot saw the leave domain button available.

So I disjoined it and then attempted to rejoin the domain while running wireshark on the NIC of the 2008R2 DC. Then I brought up a 2012R2 DC, waited 10 minutes or so then shut down the 2008R2. Again I ran wireshark but this time on the NIC of the 2012R2 server. I compare the results of a successful and unsuccessful join and saw the 2012R2 DC had a lack of SMB packets outbound to the VCSA.The server service was running on the 2012 DC, but Microsoft's recommendation was to change the srv.sys startup from auto to manual on server 2012 R2. I thought that this was odd at the time, but we changed it when we set up the new DCs to satisfy the BPA. I compared it to the registry setting on the 2008R2 DC. which was set to auto start.

So to allow VCSA to join the domain when you are getting the 31 error, undo this

http://social.technet.microsoft.com/wiki/contents/articles/21104.srv-sys-should-be-set-to-start-on-d...

on your server 2012R2 DCs if it has been implemented as it causes the domain join to fail.

Command prompt: sc config srv start=auto

or

Registry: HKLM:\System\CurrentControlSet\Services\srv\ from 3 to 2


Hope this saves someone else the amount of time it took me to figure it out! I guess I can cancel my support request now...


ds5384
Contributor
Contributor
Jump to solution

Thank you, WhiskyTangoFoxtrot

Worked with VMWARE for months attempting to resolve this issue. Of course it was the domain controller. This fix worked after changing these settings and rebooting the DC. Truly, nice work.

Reply
0 Kudos
OliverHetzner
Contributor
Contributor
Jump to solution

Thanks a lot,

you made my day!

Reply
0 Kudos
Trucaliber
Contributor
Contributor
Jump to solution

Thank you for your thorough troubleshooting and post.  This lead me straight to the issue at hand for adding VCSA 5.5 to a Windows 2012R2 domain.   My 2012R2 AD server wasn't even running SRV and I couldn't find it in the registry.  Only SRV2.sys was present.  My fix was to add the "SMB 1.0/CIFS File Sharing Support" feature.  Once added the joindomain-cli command worked perfectly.

Win2012R2-SRV-SMBv1.png

Reply
0 Kudos
ChrisKuhns
Enthusiast
Enthusiast
Jump to solution

You sir, are my new GD hero.

Reply
0 Kudos
douglasarcidino
Hot Shot
Hot Shot
Jump to solution

Sounds like someone with a support agreement needs to do a feature request for SMB2 support on the VCSA.

Doug

If you found this reply helpful, please mark as answer VCP-DCV 4/5/6 VCP-DTM 5/6
Reply
0 Kudos
SteveGalbincea
VMware Employee
VMware Employee
Jump to solution

Yes yes yes! Thank you so much for this, we have struggled with this one for some time!

Reply
0 Kudos
GeorgeStrother
Contributor
Contributor
Jump to solution

How to turn SMB2 on the vCSAs:

SSH into the vCSA and run:

/opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]' Smb2Enabled 1

You can verify the values with the following command:

/opt/likewise/bin/lwregshell list_values '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]'

Then restart likewise:

/opt/likewise/bin/lwsm restart lwio

Now it talks to AD with SMB2

George

bsd1977jda
Contributor
Contributor
Jump to solution

I tried GeorgeStrother's solution and it worked for me.  IMO this is a much less intrusive action than enabling SMBv1 on all of my domain controllers. 

Does anyone know why SMBv2 should not be enabled on the VCSA (or external PSC)?

If there are not any drawbacks I think there should be a KB article on this fix instead of the SMBv1 fix on the DC's.

Reply
0 Kudos
frostyk
Enthusiast
Enthusiast
Jump to solution

Thanks for sharing the solution.  I had two vCenters have this issue and it fixed both.  Yaa for active directory teams not telling people or researching what uses SMB1.

Reply
0 Kudos