1 Reply Latest reply on Jun 13, 2020 7:00 AM by relmes

    vCenter 6.7 generating ICMP port unreachable for DNS query responses

    rsmith42 Lurker

      Hello folks,

       

      I have two installations with vCenter installed on them that are both doing something that seems peculiar.  They send DNS queries from the vCenter IP address to the assigned DNS servers.  Roughly 15% of the time, they will generate a query the same second with the same source port to both the primary and secondary DNS servers.  When this happens, the firewall logs that the vCenter server responds to the secondary DNS server's answer with an ICMP type 3 code 3 (port unreachable).  This clearly shouldn't be happening (the query to the secondary DNS server probably shouldn't even be happening).  Any ideas on why this would occur?

       

      vCenter Server with an embedded Platform Services Controller

      v6.7.0.30000

      Build 13010631

       

      Example firewall log entries:

      access-list vm_interface_access_in permitted udp vm-interface/10.1.1.5(33201) -> dc_interface/10.1.3.1(53)

      access-list vm_interface_access_in permitted udp vm-interface/10.1.1.5(33201) -> dc_interface/10.1.3.2(53)

      No matching connection for ICMP error message:  icmp src vm_interface:10.1.1.5 dst dc_interface:10.1.3.2 (type 3, code 3) on vm_interface.  Original IP payload:  udp src 10.1.3.2/53 dst 10.1.1.5/33201.

       

      Thank you!

        • 1. Re: vCenter 6.7 generating ICMP port unreachable for DNS query responses
          relmes Lurker

          Same issue here.  Surely vCenter shouldn't be sending out two different DNS requests from the same ephemeral port before the first request has been replied to.

          My assumption is that it is closing the port on vCenter once it gets the first response (from which ever of the two responds quickest), so that the second response hits a closed port and then vCentre sends the ICMP port unreachable.

           

          Packet Capture and related log of an example instance from me:

           

          3168: 13:41:37.191045   802.1Q vlan#111 P0 10.216.20.28.36228 > 10.200.223.6.53:  udp 58
          3169: 13:41:37.191152   802.1Q vlan#111 P0 10.216.20.28.36228 > 10.200.224.6.53:  udp 58
          3170: 13:41:37.192739   802.1Q vlan#111 P0 10.200.224.6.53 > 10.216.20.28.36228:  udp 128
          3171: 13:41:37.193639   802.1Q vlan#111 P0 10.200.223.6.53 > 10.216.20.28.36228:  udp 128

           

          Jun 13 2020 13:41:37 %ASA-4-313005: No matching connection for ICMP error message: icmp src oob-lzb:10.216.20.28 dst oob-met:10.200.223.6 (type 3, code 3) on oob-lzb interface.  Original IP payload: udp src 10.200.223.6/53 dst 10.216.20.28/36228.

           

          Looks like a bug to me.

          We are running vCentre Server Appliance 6.7.0.44000 (with embedded PSC)