8 Replies Latest reply on Jul 30, 2013 2:27 PM by vmmochi

    Distributed switch problem

    timgr Lurker

      I am configuring a new VMkernel distributed switch and trying to add a new vMotion virtual adapter. I input the correct IP address information and received the error -

       

      “Call "HostNetworkSystem.AddVirtualNic" for object "networkSystem-99" on vCenter Server "" failed.

      Operation failed, diagnostics report: A vmkernel nic for the connection point already exists:”.

       

      I have tried all sorts of things but it will not add the VMkernel IP. I would be very grateful if anyone has any ideas.

        • 1. Re: Distributed switch problem
          vGuy Expert

          Do you see the vmkernel interface from the console, esxcfg-vswitch -l ?

          • 2. Re: Distributed switch problem
            timgr Lurker

            Attached is the output from the command and screen shot from VC. I am just trying to add the VMkernel IP to the VMKernel port group. The management IP is fine on the management port group.

             

            ~ # esxcfg-vswitch -l
            DVS Name         Num Ports   Used Ports  Configured Ports  MTU     Uplinks
            Production Data  256         3           256               1500    vmnic5,vmnic4

              DVPort ID           In Use      Client
              512                 1           vmnic4
              513                 1           vmnic5

            DVS Name         Num Ports   Used Ports  Configured Ports  MTU     Uplinks
            Production VMKernel  256         6           256               1500    vmnic0,vmnic3,vmnic2,vmnic1

              DVPort ID           In Use      Client
              384                 1           vmnic0
              385                 1           vmnic1
              386                 1           vmnic2
              387                 1           vmnic3
              256                 0
              128                 1           vmk0

            • 3. Re: Distributed switch problem
              vGuy Expert

              It's possible the dvs configuration is not synched to vCenter. I suggest you to try creating vmknic using CLI, below is the command for reference:

              esxcfg-vmknic -a -i IP_address -n netmask -s DVswitchname -v virtual_port_ID

               

              some additional reference: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008127

               

              Also on your Management PG, the vmk0 does not show any IP address? You might want to check the esxcfg-vmknic -l output (which should reflect the assigned IP since you're able to connect to vCenter) and restart the mgmt. services to sync the configuration with the vCenter.

              • 4. Re: Distributed switch problem
                godbucket Enthusiast

                I'm having the exact same issue. This has happened to me multiple times since I upgraded to 5.1. Anytime I make the smallest changes to a dvPort Group, an Uplink, or remove a connection then add it back, this problem occurs. VMware support said to restart the vCenter Service and that would resolve it. This does indeed work, but c'mon... restarting the vCenter Service EVERYTIME this happens?!?! Seriously? Need a bugfix/hotfix for this pronto.

                • 5. Re: Distributed switch problem
                  kashifkarar01 Enthusiast

                  I had the same experience, did a little more re-search and read about dvSwitch Sync control plane syncs with I/O plane and may get out of sync at times. (sync is at 5 min interval)

                   

                  > esx/i hosts will continue to function if vCenter is down but config update will not be possible until vCenter is restored.  Backup copy of dvSwitch is stored on esx/i hosts in /etc/vmware/dvsdata.db.

                   

                  > dvsdata.db is the file that net-dvs which was also mentioned earlier reads its information from.

                   

                  I waited for more than 5 mins and had the same issue, restarting the VC Service helps.

                  • 6. Re: Distributed switch problem
                    godbucket Enthusiast

                    Exactly my point, that's so lame. We shouldnt have to restart the vCenter service for this. We have backup jobs leveraging VMware backup API's that will crash if the service is restarted. Not a suitable workaround for us. Their dev teams should really look into this.

                     

                    Slightly off topic, but I also noticed whenever I want to delete a dvPort Group I have to do this same thing; restart vCenter Service. So lame.

                    • 7. Re: Distributed switch problem
                      jenkintl Novice

                      I experienced this same problem after moving my ESXi 5.0 U1a host back to a standard switch for some testing.  Once I moved my host back, in this case, to a 1000v it moved back fine but I had to recreate my vMotion VMkernel port and got the message "A vmkernel nic for the connection point already exists".  I sshed into the host and checked to see if the VMK1 existed and it did.  Command:  esxcli network ip interface list

                       

                      I issued the command to delete it.  Command:  esxcli network ip interface remove --interface-name=vmk1

                       

                      Once I deleted vmk1 I was able to recreate it via the client through vCenter with no issues.

                       

                      Hope this helps.

                      • 8. Re: Distributed switch problem
                        vmmochi Novice

                        Thanks, the issue came up for me in esxi 5.1.  Remove hidden vmkernel port through command line and re-add kernels and iscsi bindings.

                         

                         

                        ~ # esxcli network ip interface list

                        vmk0

                           Name: vmk0

                           Enabled: true

                           Portset: DvsPortset-2

                           Portgroup: N/A

                           VDS Name:

                         

                           VDS Port: 468

                           VDS Connection:

                           MTU: 1500

                           TSO MSS: 65535

                           Port ID: 67108871

                         

                         

                        vmk1

                           Name: vmk1

                           MAC Address:

                           Enabled: false

                           Portset: DvsPortset-0

                           Portgroup: N/A

                           VDS Name: Storagedvswitch

                           VDS UUID:

                           VDS Port: 1

                           VDS Connection:

                           MTU: 0

                           TSO MSS: 0

                           Port ID: 0

                         

                         

                         

                         

                         

                         

                        ~ # esxcli network ip interface remove --interface-name=vmk1

                         

                         

                        RECREATE ISCSI PORTS AND add them back to iscsi adapter. no need to reboot host.