VMware Cloud Community
socius
Contributor
Contributor

RDS/Terminal Server functionality stopped working after VMware-tools upgrade.

Hello, today I set out to batch upgrade VMware-tools on all Windows VMs in our environment, to patch the vulnerability described in VMSA-2016-0010.

Something very strange happened, after upgrading about 500 VMs, I started getting tickets and alarms about our Terminal Servers not working properly.

They all responded to ICMP requests, but no longer responded on RDP. Also, trying to log onto the servers through the console resulted in an error message about there not being any Domain Controllers available.

The issue was resolved by rebooting the VMs, (which is not a desirable option since our external staff using these machines have things running on them that preferably should not be stopped.)

I was wondering if this is a known error or bug and if there is a known workaround, as I have quite a few VMs left to upgrade, several of them Terminal Servers.

Environment information:

Four physical sites, nine clusters. VMs seem to be affected regardless of cluster or site.

Six clusters running ESXi 6.0U2 (6.0.0, 3620759), three legacy clusters running ESXi 5.5.0, 3568722.

VMware vCenter Server running 6.0.0, 3634794

Affected VMs are running Windows Server 2008 R2 or Windows Server 2012 R2

Upgrade was done in batches of between 50-75 VMs at a time (manually, not scripted), by marking VMs in the user interface, right clicking them and choosing Install/Upgrade VMware-tools.

The following advanced option to suppress reboot was used: /S /v"/qn REBOOT=R"

I hope this information is sufficient for helping me figure out what happened. Thanks in advance.

/socius

0 Kudos
2 Replies
RyanLiebke
Contributor
Contributor

An upgrade of the VMware Tools can update network card drivers. When this happens on RDS servers you might notice TerminalServices-Session, TerminalServices-Session-Client, Application, and System event log entries where the server is removed from the farm at the same time the network card drivers are reinstalled, then fails to rejoin the farm, and continues to generate a pair connection errors for RDP\RDS sessions until rebooted.

Events triggered at time of upgrade:

Driver Management concluded the process to install driver vmxnet3.inf....

Remote Desktop Services successfully left a farm on the Connection Broker server...

Remote Desktop Services failed to join the Connection Broker on server...

Events triggered upon connection attempt:

Error: Current async message was dropped by async dispatcher...

Remote Desktop Connection Broker Client failed while getting redirection packet from Connection Broker...

A reboot is best since its pending and can get in the way of other planned maintenance and patching later. Obviously not an ideal situation but it can be avoided by disabling auto-upgrade on power cycle, and manually upgrading tools without suppressing the reboot during scheduled maintenance windows. Just wanted to give this old post some more content in case anyone else encounters it.

0 Kudos
gw1966
Contributor
Contributor

See this workaround we had to do after upgrading to Vmware tools 10.2. All previous versions of tools were unaffected.

We had to disable CAF service via GPO

Problem with Vmware tools 10.2 and VMware CAF Management Agent Service

Grant

0 Kudos