VMware Networking Community
Mickeybyte2
Hot Shot
Hot Shot
Jump to solution

ping loss vmware tools 11

Hi,

We are running server VM's on a VMware Cluster. We are running VDI VM's (Horizon) on another VMWare Cluster running. The clusters are managed by a service provider. All I know is that they are using NSX and Trend Micro  Deep Security on the network.

We have a pool of VDI's running vmware tools 10.3.5, let's call this Pool10

We have another pool of VDI's running vmware tools 11.0.1, let's call this Pool11

We have servers running vmware tools 11.0.1.

We noticed that we have >1% ping loss when pinging between servers or between servers and Pool11

We don't have ping loss when pinging to servers from Pool10

In the release notes of VMWare tools 11.0.5 I read:

Outbound IPv6 traffic for ICMP and UDP protocols could experience packet drops.

NSX Network Introspection Driver is used to retrieve the network context from the Guest VMs.
From VMTools 11.0.0, support has been added for ICMP and UDP protocols. The driver now intercepts the UDP and ICMP traffic, collects the required context, and re-injects the packets back. There was a problem in the packet re-injection code for the ICMP and UDP IPv6 packets for outbound traffic. As a result, outbound IPv6 traffic for ICMP and UDP protocols could experience packet drops.

Note: Network Introspection Driver is not installed by default and can be installed with VMware Tools ‘Complete’ installation for NSX IDFW and NSX Intelligent features only.

This issue is fixed in this release.

We are using IPv4 only (IPv6 is not disabled, but not used)

I've tested the ping problem with VDI Pool11 running 11.0.5 and the issue is still there

I've tested the ping problem with VDI Pool10 running 10.3.5 and there were no ping losses anymore!

Is anybody here aware of a solution other than to revert all our VMWare tools to 10.3.5? (As said, I don't have experience, nor access to the NSX setup, I just know it's there)

Thanks a lot,

Michiel.

Regards, Michiel.
Tags (2)
1 Solution

Accepted Solutions
davidi123
Contributor
Contributor
Jump to solution

Ok finally VMWare found now the bug:

>This issue has been addressed in VMTools 11.1. which is due for release on 14/04/2020 (please be advised this is subject to change.).

>Meanwhile we advise you to workaround this issue by uninstalling the Network Introspection Driver if you are not using any Identity Firewall or Endpoint Monitoring feature.

View solution in original post

24 Replies
davidi123
Contributor
Contributor
Jump to solution

We have the same issue also with Tools 11.0.1/11.0.5 and TMDS -> I have opened a VMWare case

Reply
0 Kudos
joking
Contributor
Contributor
Jump to solution

We are experiencing the same issue with tools 11.0.1 and have opened a service request with VMware. The ticket was initially assigned to the networking team and we were able to narrow it down to the Guest Introspection Driver in tools 11.0.1 and 11.0.5. We are currently using Trend Micro Deep Security in our environment, which requires using the driver, so removing it is not an option. The current work around to leave the guest introspection driver enabled is to downgrade tools to version 10.3.10. The ticket has been cloned to the NSX team to troubleshoot the issue further, which I have a call scheduled today at 1:00 PM EST.

davidi123
Contributor
Contributor
Jump to solution

so you can say them it's the same problem in case: SR20104757202 (our case number).

if you want you can share youre number so i can told the same to my techie.

thanks

davidi123
Contributor
Contributor
Jump to solution

by the way:

I have another case opened. When I tracer with the new VM Tools and the NSX driver.
I see only the last hop -> because the NSX drivers faulty manipulates the TTL of the traceroute packets.

It sets the default TTL instead of using an increment of 1. therefore traceroute is broken..

Mickeybyte2
Hot Shot
Hot Shot
Jump to solution

Thanks for the update. I would really appreciate it if you could keep me updated on the outcome of the support requests.

As much as I'd like to, I'm not in the ability to open a support request unfortunately...

Regards.

Regards, Michiel.
Reply
0 Kudos
VDINinja311
Enthusiast
Enthusiast
Jump to solution

Also saw this behavior with 11.0.5 and we backed down to VMware Tools 10.3.10. Will be interested in seeing what the outcome of your tickets with VMware are. I was going to enter one in myself after I figured it out but will save myself the headache Smiley Happy

Please update with any info you all get, much appreciated!

Reply
0 Kudos
JonathanMcDowel
Contributor
Contributor
Jump to solution

We are also experiencing the same issue. It only seems to happen to windows 10 workstations I haven't had the same loss on any Server 2016 machines.

Reply
0 Kudos
ChicaneUK
Enthusiast
Enthusiast
Jump to solution

Another user experiencing this exact same issue with 11.0.5 tools, on Windows 10 desktops. A colleague found unloading the Network Introspection Driver resolves the issue for us, or alternatively stopping and disabling the vnetWFP service will immediately rectify the issue. I'd be interested to hear the output of any of these tickets with VMware.. does anyone have any updates / progress? I've opened one myself this morning.

Reply
0 Kudos
davidi123
Contributor
Contributor
Jump to solution

Ok finally VMWare found now the bug:

>This issue has been addressed in VMTools 11.1. which is due for release on 14/04/2020 (please be advised this is subject to change.).

>Meanwhile we advise you to workaround this issue by uninstalling the Network Introspection Driver if you are not using any Identity Firewall or Endpoint Monitoring feature.

Mickeybyte2
Hot Shot
Hot Shot
Jump to solution

davidi123

Thanks for the update!

Did VMware by any chance say what to do if you do need the introspection driver for endpoint monitoring? I suppose reverting to 10.3.x will be the case then? It's still a long time to wait until 11.1 will be available apparently.

Regards,

Michiel.

Regards, Michiel.
Reply
0 Kudos
ChicaneUK
Enthusiast
Enthusiast
Jump to solution

My ticket went through standard support and has now been passed to the NSX support group although my last contact from them was on Friday morning. I replied later in the same day and have yet to have a response from them, which is a little disappointing.

Reply
0 Kudos
mouraMXmouraMX
Contributor
Contributor
Jump to solution

Hey guys!

I have the exactly same environment and issues.

Did VMware provide any update?

Tks,

Reply
0 Kudos
davidi123
Contributor
Contributor
Jump to solution

no news. its clear: wait for the new tools or downgrade to 10.3 branch

mouraMXmouraMX
Contributor
Contributor
Jump to solution

Hello Guys,

Any news from VMware regarding the new release?

Reply
0 Kudos
jmatz135
Hot Shot
Hot Shot
Jump to solution

The notes specify that "There was a problem in the packet re-injection code for the ICMP and UDP IPv6 packets for outbound traffic. As a result, outbound IPv6 traffic for ICMP and UDP protocols could experience packet drops."

Presumably this means that only IPv6 traffic for ICMP and UDP connections are effected.  This doesn't really seem to be the case.  What actually is effected by the bug?  What packets are being dropped?

Reply
0 Kudos
ChicaneUK
Enthusiast
Enthusiast
Jump to solution

I notice that VMware Tools 11.0.6 is out though no mention of a fix nor any acknowledgement that it's a known issue.. which seems a bit sneaky to me!

Reply
0 Kudos
VDINinja311
Enthusiast
Enthusiast
Jump to solution

Just installed VMware Tools 11.1 (was released yesterday) and am still seeing dropped pings. Not seeing any mention of the issue in resolved bugs or known bugs in the 11.1 release notes.

VMHero4Ever
Enthusiast
Enthusiast
Jump to solution

Same for me. VMware Tools 11.1 has the same issue like the versions before.

Reply
0 Kudos
tcommander
Contributor
Contributor
Jump to solution

Has anyone tried the VMware tools from June 18 that list a fix for this issue?  We have installed it based on this issue and are beginning to test but wanted to see if anyone else had feedback on the issue.

Reply
0 Kudos