Agent Unreachable after restore VMware Horizon Connection Server

Andre_Fellipe — Thu, 31 Dec 2020 23:30:22 GMT

After a power outage I got both of my View Connection Servers unbootable with BSOD and I could not recovery it and also I don't have backup of it.

After all steps below I could not get things fixed, all VMs are "Agent Unreachable"

Created a new VM for connection server (WITH THE SAME NAME AS THE OLD "VIEWCS01")

Before doing it correctly I connected the base disk, and not the last snapshot, to the new VM and broke up the whole thing with the error "file system specific implementation of ioctl [file] failed", I solved this correcting the CID - https://kb.vmware.com/s/article/1007969

Installed Windows (Same version)
As in https://kb.vmware.com/s/article/76770:
- Installed Connection Server (Same Version)
- Restored LDF backup
- Removed all View Connection Servers and all Security Servers (vdmadmin -S -s -r viewcs01, viewss01 and viewcs02)
- Uninstalled the connection server
- Reinstalled the connection server reusing the AD LDS

I did removed the "viewcs01" because with my previous tests I was not removing it, I think because of >this, after the recovery steps done no console was opening, In previous tests I also not using the old >machine name, instead of it I was using "viewcs03".

Ok, Console opened, I changed the vCenter credentials (Just putting password was not working with error - https://kb.vmware.com/s/article/60152 - Log below:

2020-12-31T18:23:48.108-02:00 ERROR (18F8-0D0C) <MessageFrameWorkDispatch> [ws_java_bridgeDLL] BCryptDecrypt FAILED, status={Data Error}
An error in reading or writing data occurred. (0xC000003E)
2020-12-31T18:23:48.109-02:00 ERROR (18F8-0C54) <VCHealthUpdate> [SecurityManagerUtil] decryptAsText: com.vmware.vdi.crypto.SecurityManagerException: decrypt: Cannot decrypt: Cipher scheme decryption failed.
2020-12-31T18:23:48.109-02:00 DEBUG (18F8-0C54) <VCHealthUpdate> [ServiceConnection25] Connecting instance VCHealth Test instance at URL https://vcenterd.DOMAIN.net:443/sdk

Corrected Composer credentials, and added license.

All machines are "Agent Unreachable" - Connection Server Log below:

2020-12-31T18:23:49.160-02:00 DEBUG (18F8-1A6C) <DesktopControlJMS> [DesktopTracker] CHANGEKEY message from agent/bda3fbe6-029c-41f8-b9f8-017af574f56b accepted as key and thumbprints match machine record
2020-12-31T18:23:49.162-02:00 DEBUG (18F8-1A6C) <DesktopControlJMS> [DesktopTracker] found broker thumbprints: 0f:9e:80:5d:f6:33:c7:1b:a2:d5:8c:9a:9f:12:45:16:0f:6f:c0:2b:46:8d:d0:33:62:87:53:a9:48:8d:57:8c#SHA_256;51:c5:d0:44:02:7f:ca:6d:5a:ad:5b:f6:8d:f5:11:23:e8:aa:e1:91:d0:5c:ff:71:3b:fb:e2:4b:f4:12:5e:d5#SHA_256
2020-12-31T18:23:49.162-02:00 WARN  (18F8-1A6C) <DesktopControlJMS> [JMSMessageSecurity] Failed to sign message: Cannot sign message
2020-12-31T18:23:49.162-02:00 DEBUG (18F8-1A6C) <DesktopControlJMS> [DesktopTracker] CHANGEKEY message from agent/bda3fbe6-029c-41f8-b9f8-017af574f56b result: true (success)

Excerpt from VM agent log:

2020-12-31T19:53:44.322-03:00 DEBUG (1EDC-0FA8) <Thread-4> [AgentJmsConfig] Using paired signing key
2020-12-31T19:53:44.322-03:00 DEBUG (1EDC-0FA8) <Thread-4> [AgentMessageSecurityHandler] Configuring message security (ENHANCED).
2020-12-31T19:53:44.369-03:00 DEBUG (1EDC-0FA8) <Thread-4> [BrokerUpdateUtility] Published CHANGEKEY request
2020-12-31T19:53:59.386-03:00 DEBUG (1EDC-0FA8) <Thread-4> [BrokerUpdateUtility] Timeout waiting for success response
2020-12-31T19:59:33.944-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] Using connection broker viewcs01.DOMAIN.net
2020-12-31T19:59:33.944-03:00 DEBUG (1430-2494) <MessageFrameWorkDispatch> [MessageFrameWork] KeyVault service got operation=getEndEntityCertificates, ok=1, msecs=0
2020-12-31T19:59:33.944-03:00 DEBUG (1430-2494) <MessageFrameWorkDispatch> [MessageFrameWork] KeyVault service got operation=getEndEntityCertificates, ok=1, msecs=0
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] username for swiftmq connection is: agent/90916ab8-704c-4fe3-a605-c4a7745b246e
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [AgentJmsConfig] Skipping pair operation: already paired
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [AgentMessageSecurityHandler] Configuring message security (ENHANCED).
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] Re-connecting using secure port 4002
2020-12-31T19:59:34.381-03:00 DEBUG (1430-2780) <SwiftMQ-ConnectorPool-2> [AgentSSLSocketFactory] Received cert with subject cn=router/viewcs01
2020-12-31T19:59:34.381-03:00 WARN  (1430-2780) <SwiftMQ-ConnectorPool-2> [AgentSSLSocketFactory] Certificate thumbprint verification failed, no matching thumbprint. Presented identity: router/viewcs01
2020-12-31T19:59:34.381-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] Unable to connect to JMS server viewcs01.DOMAIN.net com.vmware.vdi.logger.Logger.debug(Logger.java:44)
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Unexpected certificate: router/viewcs01
2020-12-31T19:59:34.381-03:00 WARN (1430-2558) <Thread-4> [JmsManager] Unable to connect to any listed host. The agent will continue to retry: [viewcs02.DOMAIN.net, viewcs01.DOMAIN.net]

Reinstalled the agent and also tried the command below, as mentioned in https://kb.vmware.com/s/article/2038679, nothing has worked at all.

vdmadmin -A -d desktop-pool-name -m name-of-machine-in-pool -resetkey

Re: Agent Unreachable after restore VMware Horizon Connection Server

heimdm — Wed, 16 Feb 2022 18:15:48 GMT

Did you ever get a solution on this?

Re: Agent Unreachable after restore VMware Horizon Connection Server

Andre_Fellipe — Wed, 16 Feb 2022 18:38:56 GMT

Unfortunately no, What I did as solution was creating a new connection server from scratch, creating a manual pool and adding all machines to it.

I've used the recovered one to export the assigned users list and assigned it manually for 150 VDIs.

After everything was working I started to migrate them to new pools.

topic Re: Agent Unreachable after restore VMware Horizon Connection Server in Horizon Desktops and Apps

Agent Unreachable after restore VMware Horizon Connection Server

Re: Agent Unreachable after restore VMware Horizon Connection Server

Re: Agent Unreachable after restore VMware Horizon Connection Server