After upgrading from Horizon 8 2111 to 2206 - We get on Con.Server page "An Internal Error Occurred"

[L3-MAS]

After upgrading from Horizon 8 2111 to 2206 - We get on some page "An Internal Error Occurred"

To reproduce it on the Admin Portal of Horizon 8 2206

Under Settings - Servers - Connection Servers and we get "An Internal Occrurred". 

So far, its seems the only pages that returns this errors

However, under Monitor - Dashboard - My 4 Connections Servers are there reporting no issues.

Debug Logs on Connections Servers return this when we click on - Connections Servers -

2022-09-09T07:30:02.514-04:00 ERROR (1070-1AE4) <ajp-nio-127.0.0.1-8009-exec-1> [FaultUtilBase] InvalidArgument(parameter: connectionServers): A null value is invalid.
2022-09-09T07:30:02.514-04:00 DEBUG (1070-1AE4) <ajp-nio-127.0.0.1-8009-exec-1> [VlsiInvocation] Finish to call view API for /view-vlsi/rest/v1/connectionserver/list elapsed time:7 msecs
2022-09-09T07:30:02.514-04:00 ERROR (1070-1AE4) <ajp-nio-127.0.0.1-8009-exec-1> [RestApiServlet] Unexpected fault:(vdi.fault.InvalidArgument) {
errorMessage = A null value is invalid.,
parameterName = connectionServers
} for uri /view-vlsi/rest/v1/ConnectionServer/list
2022-09-09T07:30:02.529-04:00 DEBUG (1070-1AE4) <ajp-nio-127.0.0.1-8009-exec-1> [RestApiServlet] Exception for uri /view-vlsi/rest/v1/ConnectionServer/list com.vmware.vdi.logger.Logger.debug(Logger.java:44)
com.vmware.vdi.vlsi.rest.exceptions.ViewServiceFaultException: (vdi.fault.InvalidArgument) {
errorMessage = A null value is invalid.,
parameterName = connectionServers
}
at com.vmware.vdi.vlsi.rest.reflect.impl.VlsiInvocation.invoke(VlsiInvocation.java:96)
at com.vmware.vdi.vlsi.rest.reflect.ReflectEngine.invokeService(ReflectEngine.java:79)
at com.vmware.vdi.vlsi.rest.RestApiServlet.doPost(RestApiServlet.java:145)
at com.vmware.vdi.vlsi.rest.RestApiServlet.doGet(RestApiServlet.java:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:655)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.FailedRequestFilter.doFilter(FailedRequestFilter.java:97)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.vmware.vdi.vlsi.rest.EncodingFilter.doFilter(EncodingFilter.java:37)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.vmware.vdi.vlsi.rest.authentication.RestApiAuthFilter.doFilter(RestApiAuthFilter.java:143)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:263)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:367)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:526)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:882)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1691)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: (vdi.fault.InvalidArgument) {
errorMessage = A null value is invalid.,
parameterName = connectionServers
}
at com.vmware.vdi.vlsi.server.util.FaultUtilBase.createInvalidArgument(FaultUtilBase.java:551)
at com.vmware.vdi.vlsi.server.vdi.infrastructure.GSSAPIAuthenticatorUtil$GSSAPIAuthenticatorInfoUtil.setConnectionServers(Unknown Source)
at com.vmware.vdi.vlsi.server.vdi.infrastructure.GSSAPIAuthenticatorUtil$GSSAPIAuthenticatorInfoUtil.setMember(Unknown Source)
at com.vmware.vdi.vlsi.server.vdi.infrastructure.GSSAPIAuthenticatorUtil$GSSAPIAuthenticatorInfoUtil.read(Unknown Source)
at com.vmware.vdi.vlsi.server.infrastructure.GSSAPIAuthenticatorManager.populate(GSSAPIAuthenticatorManager.java:99)
at com.vmware.vdi.vlsi.server.infrastructure.GSSAPIAuthenticatorManager.list(GSSAPIAuthenticatorManager.java:205)
at com.vmware.vdi.vlsi.server.infrastructure.GSSAPIAuthenticatorManager.getGssapiAuthenticationBycsId(GSSAPIAuthenticatorManager.java:267)
at com.vmware.vdi.vlsi.server.infrastructure.ConnectionServerManager.getCSInfo(ConnectionServerManager.java:326)
at com.vmware.vdi.vlsi.server.infrastructure.ConnectionServerManager.fetchConnectionServers(ConnectionServerManager.java:247)
at com.vmware.vdi.vlsi.server.infrastructure.ConnectionServerManager.list(ConnectionServerManager.java:280)
at com.vmware.vdi.vlsi.server.infrastructure.ConnectionServerServer.list(ConnectionServerServer.java:51)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.vmware.vdi.vlsi.rest.service.impl.VlsiMethodWrapper.invoke(VlsiMethodWrapper.java:49)
at com.vmware.vdi.vlsi.rest.reflect.impl.VlsiInvocation.invoke(VlsiInvocation.java:81)
... 34 more
2022-09-09T07:30:02.529-04:00 DEBUG (1070-1AE4) <ajp-nio-127.0.0.1-8009-exec-1> [RestApiAuthFilter] Send response for /view-vlsi/rest/v1/ConnectionServer/list elapsed time:23 msecs

If any one can help it will be apprecated !

[vBritinUSA]

Hi @L3-MAS 

Check timing between the Connection Servers and also check that the Adam DB is in good health and force a replication.

Forcing replication between ADAM databases (1021805) (vmware.com)

The Event log of the ADAM database in VMware Horizon View reports the error: Ownership of the follow...

[L3-MAS]

Hi,

We do have 4 connections servers in total. When we check the replications between the 4 servers, for each connection server the inbound neighbor only 2 of them instead of 3.

For instance :

Connection Server #1 have as inbound neightbor CS #2 and CS #4 which the replication are successful

Connection Server #2 have as inbound neightbor CS #3 and CS #4 which the replication are successful

Connection Server #3 have as inbound neightbor CS #1 and CS #2 which the replication are successful

Connection Server #4 have as inbound neightbor CS #2 and CS #3 which the replication are successful

I dont know if its normal to have only 2/3 CS as inbound neightbor for all of them, but all of them are well replicated successfully with their 2 inbound neigtbor.

For ADAM Event logs, warning about backup interval time, nothing else

This directory partition has not been backed up since at least the following number of days.

Directory partition:
CN=Configuration,CN={3813E705-A030-4E75-9B77-475A997E689E}

'Backup latency interval' (days):
90

It is recommended that you take a backup as often as possible to recover from accidental loss of data. However if you haven't taken a backup since at least the 'backup latency interval' number of days, this message will be logged every day until a backup is taken. You can take a backup of any replica that holds this partition.

By default the 'Backup latency interval' is set to half the 'Tombstone Lifetime Interval'. If you want to change the default 'Backup latency interval', you could do so by adding the following registry key.

'Backup latency interval' (days) registry key:
System\CurrentControlSet\Services\NTDS\Parameters\Backup Latency Threshold (days)

[L3-MAS · Accepted solution]

Problem Solved

After I did analyze in ADLDS (ADAM) the 4 connections servers "CN Servers Objects" under OU=Server,OU=Properties,DC=vdi,DC=vmware,DC=int,  I found that 2 of them didn't have the attribute -  pae-GSSAPIConfigDN - populated with an Authenticator CN Objects that reside in OU=Authenticator,OU=Properties,DC=vdi,DC=vmware,DC=int

Comparing all CN Authenticator Object one by one, I did figured out 2 of them had also 2 attibutes - pae-GSSAPIConfigDNOf - not set.

So on the CN Server Object side I add the CN Authenticator DN string and on the CN Authenticat Object side I add the CN Server DN String.

Ex : 

Then I refresh my Horizon Admin Portal page and I got the connection servers back and responding

I hope this can help someone else

Eric

[vBritinUSA]

Nice find and thanks for sharing!

[Ray_handels]

Hey Eric,

We have the exact same issue and I also see that the record pae-GSSAPIConfigDN for the servers are not configured for 4 of the servers. When looking at the OU=Authenticator,OU=Properties,DC=vdi,DC=vmware,DC=int settings I see 6 records, 5 of which are filled with information of 2 of the connection servers (which also have the pae-GSSAPIConfigDN record in it) but 4 connection servers don't seem to have any information in both options. 

Did you have any reference between the servers and the authenticator to be able to fill in the correct information?

Regards,

Raymond