VMware Cloud Community
asg2ki
Enthusiast
Enthusiast
Jump to solution

Webclient browsing problem with multi-site PSC & VCS implementation

Hi all,

I just deployed a brand new VC6 environment which consist from two sites. I wanted to implement separate VC Server + External PSC instances at each site where the PSC's are sharing a common domain, so effectively the two PSC's are replicating all information between each other and the VC's are connected individually to the PSC's at each site.

My primary site seems to work absolutely fine when I logon there through the webclient, but on the second site I'm unable to browse properly anything (again via the webclient). Sometimes I receive error messages that http://localhost:10080/invsvc is not available and within the "Administration" tab I have no access to anything despite that I already gave permissions to my account. The only thing I'm getting back is the "You do not have permissions to view this object or this object does not exist" message.

After some hassle around this issue, I tried to repoint the second VC server to the first site's PSC which effectively fixed the issue, but when I switched back the secondary VC to its intended secondary PSC, the problem came back immediately. Now I also noticed that this happens only when I browse the second VC with specific accounts. I have no issues while browsing the environment with the built-in administrator@vsphere.local or even the Domain Admin account from my own internal AD "administrator@mydomain.local", but if I logon with my own admin credentials, the problem seems to persist. I assigned temporarily my AD user account to both VC's with full permissions and I had no issues browsing at both sites.

Also I'm not sure if this is a problem related to AD nested groups (I usually delegate the access via security groups in my VC servers), but I couldn't browse properly the webclient even after I assigned my own account directly to the VC instances. Again this happens only on the secondary site. I gave my account all necessary permissions to both VC's and I have no issues to browse them through the primary VC webclient, but it just doesn't go right through the second VC.

To me this seems to be some sort of PSC and/or WebClient bug. I already tried browsing the individual VC's via the regular c# vSphere client and there are no problems with the permissions at all, so effectively I do have access granted.

My environment is 100% windows based and the VMware part consist from several individual components like PSC and VCS instances being deployed on separate VM's at each site (so 4 VM's in total). I have 2 DC's at each site with healthy replication and currently I'm using SQL server 2014 databases hosted directly on the VC's instances.

Any help would be very much appreciated.

Thanks.

1 Solution

Accepted Solutions
LSchickWF
Contributor
Contributor
Jump to solution

Update - worked with support today and they had me switch my Identity Source from "Active Directory (integrated Windows Authentication)" to "Active Directory as an LDAP Source" problems went away. Next week I will add another remote site to test and see if everything still works.

View solution in original post

Reply
0 Kudos
14 Replies
asg2ki
Enthusiast
Enthusiast
Jump to solution

I managed to resolve the problem by myself after making some desperate tries with various things. Now it turns out that this was somehow related to my own AD account because I noticed that any other newly created account doesn't have the same problem. So what I did as a simple blind try was to clone my own account and... voila, my problems with the permissions stopped immediately. I'm absolutely not sure what was happening with my originally created account because my environment is pretty clean including AD itself, but somehow vCenter didn't like my account for whatever reason.

Furthermore I removed my old account and renamed my new one to have literally the same settings as before (including all security group membership assignments), and so far I'm not experiencing the issue anymore. The only thing I'm curious if vCenter or PSC has some sort of bug regarding the creation time of a particular account, but that's not likely to be the case simply because I had no problems with the built-in administrator account. I was able to browse with it the whole vCenter environment without any issues.

Anyway in case you might be experiencing the same problems then perhaps you can first try out creating a brand new account and see if you are experiencing issues with it. If you don't then probably you should also try cloning your own account and check the results against it as well. This certainly helped in my case.

Cheers

asg2ki
Enthusiast
Enthusiast
Jump to solution

Unfortunately it seems the same problem happened again this time with my newly re-created account on another brand new VMware environment which I deployed in parallel to the existing one. The new deployment consist of it's own set of PSC and VCS servers. As part of experimenting with things I tried logging in with my old account to this brand new environment and I had no issues with it at all. This leads me to think that there is a serious bug in the WebClient part which affects the first AD account that is logging into a distributed VMware environment but of course this is yet to be confirmed.

So re-creating your account could provide a temporary workaround but in the long run it doesn't resolve anything in case you are expected to deploy more VMware instances.

Let's hope VMware tech support will see this thread and will take a look into the details.

Cheers.

Reply
0 Kudos
LSchickWF
Contributor
Contributor
Jump to solution

I am experiencing the same issue. I have 3 sites I am setting vSphere 6.0 up in and have rebuilt the environment hoping to fix this. It didn't. During the first install my first site worked as expected but the 2 remote sites behaved exactly as you stated. I saw this post about trying a new AD account and it worked, but was not a long term solution for our environment. I wiped the config clean and reinstalled. The second time Sites 1 and 2 worked as expected, but my third site is still not working. I can log into site 3 with an AD account and I have permissions to the SSO settings. I can see the permissions and make changes, but it does not let me access any other part of the webclient. Eventually I have 5 total sites to add so I am hoping to find a fix soon.

Reply
0 Kudos
asg2ki
Enthusiast
Enthusiast
Jump to solution

Just for the record, I created a VMware SR regarding this issue. The tech support guys should be analyzing some of my vCenter logs by now, so let's hope they can find a solution to this issue soon enough.

Reply
0 Kudos
LSchickWF
Contributor
Contributor
Jump to solution

Have you heard anything from VMWare Support? I've tried upgrading to the related release along with implementing the two "fixes" I've found online: Deleting and reading permissions and double checking the environment path on the servers. Nothing has corrected it yet.

Reply
0 Kudos
asg2ki
Enthusiast
Enthusiast
Jump to solution

VMware is still troubleshooting the issue. They found some misbehavior which as per their technician looks similar to what is described here (http://kb.vmware.com/kb/2125229) but I didn't find any similarities in my environment, so we are still waiting for the appropriate resolution.

FYI, I just uploaded an additional set of support bundle logs to VMware (as per their request), so let's hope soon enough there will be some response and perhaps a solution.

Reply
0 Kudos
LSchickWF
Contributor
Contributor
Jump to solution

VMWare is still troubleshooting for me as well. We have rebuilt the system step by step per Support and the first site everything works as expected, but as soon as we add the second site we loose permissions for domain accounts on both sites. We can still log in with the fat client with our domain accounts but the webclient does not work.

I found the article you listed before as well and it did not apply to my environment either. Support told me they "may" have seen this issue in other environments and there "may" be an issue with the web client.

Reply
0 Kudos
asg2ki
Enthusiast
Enthusiast
Jump to solution

Just a quick update...

I've requested a case escalation today (after being advised by the VMware tech. guy), so I hope to get contacted by senior tech. support within the next day or so. Perhaps there will be some live troubleshooting at some point, but we shall see about that in time. In any case I've been exporting quite a number of log bundles recently and it seems that the issue just can't be localized easily. This little pesky issue seems to be giving serious headaches even to VMware as far as I can see. Anyway I'll post some updates once I get progress on this one.

Reply
0 Kudos
thakala
Hot Shot
Hot Shot
Jump to solution

I am also experiencing issues with multi-site PSC deployment in my vSphere lab.

When connectivity between sites is OK Web Client works about as good as it can, but when either of sites go down browsing remaining vCenter Server with Web Client becomes horribly slow and often just result in timeout error.

It seems that Web Client cannot handle any connectivity issues between multiple vCenter Servers in multi-site deployment. This is very serious issue as one of use-cases for multi-site PSC deployment is with SRM, but it seems that if you lose your primary site you also lose management capabilities of your DR site!

Tomi http://v-reality.info
Reply
0 Kudos
LSchickWF
Contributor
Contributor
Jump to solution

Update - worked with support today and they had me switch my Identity Source from "Active Directory (integrated Windows Authentication)" to "Active Directory as an LDAP Source" problems went away. Next week I will add another remote site to test and see if everything still works.

Reply
0 Kudos
asg2ki
Enthusiast
Enthusiast
Jump to solution

So I tried the LDAP suggestion too and it work for me, so now I'm able to login properly with both my old and my new accounts.

I'm going to test few more things before I announce this internally as the ultimate solution, but all in all it seems that the problem is related specifically to the "native" AD integration way.

I'll get some very specific tests with both WebClient and C# client, but for the time being things look pretty stable with LDAP AD.

Reply
0 Kudos
jftwp
Enthusiast
Enthusiast
Jump to solution

Here we are, a year later, and in the Update 2 flavor, and we are seeing the same (or very similar) issues with web client failing to provide the expected rights while the thick client is absolutely fine with rights.  Very similar deployment model (external PSC).  Both are appliances in our case.

Switching from AD Integrated to LDAP as identity source into the same AD domain made no difference for us, unfortunately.

Case opened/escalated with VMware as this appears to be an issue for some deployments from the GA of 6.0 (when this thread began) up through the current timeframe (6.0 U2).  This only affects SOME accounts, but not all.  Only sure-fire workaround is to log on using a native SSO account outside of AD that has global rights---this is currently the only way we can confidently access all solution user/extensions such as vSphere Replication or SRM and other 'web client only' functionality.  Very disturbing problem here, and a fundamental flaw with the web client's authentication versus that of the thick client, absolutely.  Will see what the escalation yields.

Reply
0 Kudos
Rudedawg17
Contributor
Contributor
Jump to solution

Please let me know the outcome to this.  I am in the exact same boat but I have 6 sites.  I hope VMware will correct this issue with the web client especially since they are no longer supporting, and getting rid of, the thick client.

Reply
0 Kudos
mark49808
Enthusiast
Enthusiast
Jump to solution

jftwp do you have any update on this? I think i might be seeing something similar and currently going through the usual support calls but so far not getting far.

Reply
0 Kudos