8 Replies Latest reply on Dec 17, 2012 8:48 PM by JohnOCFII

    Duplicate network traffic from guest

    jahyde Novice

      When ever I ping something on the local LAN (or WAN), the replies are redirects from the HOST IP, then 4 duplicates of the reply packet from the correct IP. If I ping the host IP, they reply just fine. I am pretty sure something is wrong with the SBS nic and VMware bridge protocol.

       

      The Host is SBS2008 64bit (i know its not supported, but it works at another site), Guest: CentOS 5.4 32bit.

       

      Sample:

      SBS: 192.168.1.10 (1 NIC, win firewall off)

      CentOS: 192.168.1.9 (bridged)

      Some other PC: 192.168.1.11

       

      root@svr2:~ $ ping 192.168.1.10 (SBS)

      PING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.

      64 bytes from 192.168.1.10: icmp_seq=1 ttl=128 time=0.169 ms

      64 bytes from 192.168.1.10: icmp_seq=2 ttl=128 time=0.273 ms

      64 bytes from 192.168.1.10: icmp_seq=3 ttl=128 time=0.323 ms

       

      root@svr2:~ $ ping 192.168.1.11 (some pc)

      PING 192.168.1.11 (192.168.1.11) 56(84) bytes of data.

      From 192.168.1.10: icmp_seq=1 Redirect Network(New nexthop: 192.168.1.11)

      64 bytes from 192.168.1.11: icmp_seq=1 ttl=64 time=0.880 ms

      64 bytes from 192.168.1.11: icmp_seq=1 ttl=63 time=0.951 ms (DUP!)

      64 bytes from 192.168.1.11: icmp_seq=1 ttl=64 time=1.46 ms (DUP!)

      64 bytes from 192.168.1.11: icmp_seq=1 ttl=63 time=1.46 ms (DUP!)

      From 192.168.1.10: icmp_seq=2 Redirect Network(New nexthop: 192.168.1.11)

      64 bytes from 192.168.1.11: icmp_seq=2 ttl=64 time=1.02 ms

      64 bytes from 192.168.1.11: icmp_seq=2 ttl=63 time=1.13 ms (DUP!)

      64 bytes from 192.168.1.11: icmp_seq=2 ttl=64 time=1.51 ms (DUP!)

      64 bytes from 192.168.1.11: icmp_seq=2 ttl=63 time=1.51 ms (DUP!)

      From 192.168.1.10: icmp_seq=3 Redirect Network(New nexthop: 192.168.1.11)

      64 bytes from 192.168.1.11: icmp_seq=3 ttl=64 time=0.997 ms

      64 bytes from 192.168.1.11: icmp_seq=3 ttl=63 time=1.09 ms (DUP!)

      64 bytes from 192.168.1.11: icmp_seq=3 ttl=64 time=1.45 ms (DUP!)

      64 bytes from 192.168.1.11: icmp_seq=3 ttl=63 time=1.45 ms (DUP!)

      --- 192.168.1.11 ping statistics ---

      3 packets transmitted, 3 received, +9 duplicates, 0% packet loss, time 3109ms

      rtt min/avg/max/mdev = 0.849/1.187/1.445/0.220 ms

       

      9 DUPLICATES!!! - What is that about?

        • 1. Re: Duplicate network traffic from guest
          jahyde Novice

          I reinstalled the VM from scratch, and installed the vm-tools, and even followed the recommendation there to update the nic to the vmxnet driver.

           

          but still duplicate packets, i am almost sure its due to the OS.

          • 2. Re: Duplicate network traffic from guest
            jahyde Novice

            i configured the VM to use VMNET8 and do NAT, the problem goes away, but this will not work for me since it is supposed to be a backup for a real server on the network - it must be able to assume the address of the real server in case of a failure, meaning no nat.

             

            I have reason to believ it is because of the VMware bridge protocol in windows, or windows firewall service - which i have disabled on all interfaces, but is still running as a service.

             

            I would love it if anyone has any ideas.

            • 3. Re: Duplicate network traffic from guest
              itsgnd Novice

              I have been having the same problem.  I am using VMWare player 3.1.3 build-324285 running a Ubuntu client on a Windows 7 host.  I have tried several means of trying to determine what exactly the problem is and have found this:

               

               

              Environment:

               

              1 - Ubuntu guest running in the above configuration (192.168.0.105), called server A, MAC address 40:00:00:00:00:01

              1 - Physical Ubuntu server (192.168.0.107), called server B , MAC address 00:02:b3:f0:b7:51

              1 - Physical Windows 7 VMWare Player host machine, MAC address  90:e6:ba:80:07:f4

              Both physcal servers are on the same physical switch and VMWare is set up for bridging.

               

              Procedure:

               

              Initiate one ping packet from  server A (example: ping -c 1 192.168.0.107) to server B

              Capture all ICMP traffic on the ethernet interface using tcpdump (example: tcpdump -w capture.cap icmp) on server B

               

              Results from Wireshark analysis:

               

              Time

              Prot.

              Source

              Src MAC

              Destination

              Dst MAC

              Info

              0.00000

              ICMP

              192.168.0.105

              90:e6:ba:80:07:f4

              192.168.0.107

              00:02:b3:f0:b7:51

              Echo (ping) request

              0.00002

              ICMP

              192.168.0.107

              00:02:b3:f0:b7:51

              192.168.0.105

              40:00:00:00:00:01

              Echo (ping) reply

              0.00002

              ICMP

              192.168.0.105

              40:00:00:00:00:01

              192.168.0.107

              00:02:b3:f0:b7:51

              Echo (ping) request

              0.00003

              ICMP

              192.168.0.107

              00:02:b3:f0:b7:51

              192.168.0.105

              40:00:00:00:00:01

              Echo (ping) reply

              0.00019

              ICMP

              192.168.0.102

              90:e6:ba:80:07:f4

              192.168.0.107

              00:02:b3:f0:b7:51

              Redirect (Redirect for network)

               

                • I did validate that this pattern continues if two or more consecutive pings are initiated as well.

                • In the 1st packet the Source MAC address should be 40:00:00:00:00:01

               

              Analysis: 

               

              It appears that somehow the host machine is duplicating the ping request thereby doubling the amount of ping replies.  In addition it is causing the host to send  an ICMP redirect further polluting the network with unnecessary traffic.  Somewhere the packet is getting duplicated, I believe the most likely candidate is the windows IP stack (or maybe within the routing engine) or a bug in the VMWare driver.  It is interesting to note that the incorrect packet is sent first.

               

              The redirect seems to be an effect of the duplicate packet (the 1st ping request) and the fact that it is not originating from the appropriate MAC address associated to 192.168.0.105 in the arp table on the vmware host thereby looking like an inefficiently routed packet.

               

              Any thoughts as to how to get Windows/VMWare to stop duplicating the packet?   The redirect is a symptom, not the cause.  I have disabled redirects in the registry and filtered them out with firewall rules, but the best that is achieved is that the redirect packet is eliminated.  The duplicate packets still occur.

              • 4. Re: Duplicate network traffic from guest
                miguelpinheiro201110141 Lurker

                Hi all,

                 

                 

                I have the same problem, but now I have found a realy logic reason for that (http://en.wikipedia.org/wiki/ICMP_Redirect_Message) and a comment showing how to disable this feature in Windows (http://www.xnews.ro/QWDisable_ICMP_Redirect.htm).

                 

                I am going to try it rigth now.

                 

                hope this helps,

                miguelpinheiro

                • 5. Re: Duplicate network traffic from guest
                  layer4down Lurker

                  @itsgnd

                   

                  Thanks for your analysis of this. As a VMWARE n00b, I'm not particularly keen on the programs behaviors. I noticed that I had neglected to disable all unnecessary protocols and bindings on the VMware bridged interface of my host machine (I bridged VMnet3 to QUAD-NIC-1 and VMnet4 to QUAD-NIC-2), and my issue seems to have disappeared. I was able to verify this with wireshark as well. So my advice to anyone who comes across this issue again is to disable protocols on VMware bridged interfaces of the host machine (I left only VMware Bridge Protocol running). While this doesn't necessarily help with determining the root cause, it may help to prevent this undesired (though possibly inherent) behavior.

                   

                  Best!

                   

                  -l4d

                  • 6. Re: Duplicate network traffic from guest
                    layer4down Lurker

                    BTW-

                     

                    In case this helps someone else reading this down the road, I'd totally forgotten that I have IP Routing enabled on my Win7 box which is being affected. I do not want to disable this since I'm running OpenVPN as an internet gateway, but someone else may not need this function. I imagine that since the local physical NIC is on the same broadcast domain as the originating and destination devices, it's seeing these packets and (per protocol) retransmitting them to the (default) gateway, then sending an ICMP redirect message to the sender notifying them of such. But as someone mentioned earlier, disabling ICMP redirects does not stop the duplicate traffic. It's possible there's a function in the host OS (maybe registry?) to disable retransmission of that traffic, but I'm not sure that would be desireable; also, that would be far beyond my skillset.

                    • 7. Re: Duplicate network traffic from guest
                      dlmiles Lurker

                      For me this issue was caused by Routing and Remote Access being enabled with the service Properties and in the IPv4 tab having the "Enable IPv4 Forwarding" enabled.  There is IPv6 with a similar option to look at.

                       

                      My use case has Routing and Remote Access enabled to allow VPN endpoint but there is no through / forwarded traffic by the server all traffic and users of the VPNs terminate their traffic on the server.  So I can disable this option without concern.  Your use case maybe different if you use the VPN as a gateway onto the LAN for remote user, maybe you should consider moving the VMware guest to another physical system.

                      • 8. Re: Duplicate network traffic from guest
                        JohnOCFII Novice

                        Thanks for taking the time to post this.  This turned out to be my issue too.  For now, I turned off inbound VPN to the Host as it was not being used.