3 Replies Latest reply on Sep 14, 2019 11:21 PM by ashishsingh1508

    Can't add 10GBe physical nic

    ElliottN Lurker

      I've been having a dog of a time trying to add a dual 10GBe nic to ESXi.  I've tried installing the drivers and everything appears that it is installing but ESXi doesn't seem to be loading the drivers for some reason.  This is an Intel NUC NUC7i5BNK host with a OWC Mercury Helios 3  Thunderbolt 3 PCIe Expansion Chassis housing the NIC.  The NIC and the expansion chassis is known good on other OS's so everything points to a configuration issue on ESXi or in the bios of the NUC.  I've enabled thunderbolt at boot so I think that rules out the BIOS.

       

      [root@esxi1:~] lspci -v | grep -A1 -i ethernet

      0000:00:1f.6 Ethernet controller Network controller: Intel Corporation Ethernet Connection (4) I219-V [vmnic0]

               Class 0200: 8086:15d8

      --

      0000:06:00.0 Ethernet controller Network controller: Intel(R) 82599 10 Gigabit Dual Port Network Connection

               Class 0200: 8086:10fb

      --

      0000:06:00.1 Ethernet controller Network controller: Intel(R) 82599 10 Gigabit Dual Port Network Connection

               Class 0200: 8086:10fb

      [root@esxi1:~] vmkload_mod -l |grep drivername

      [root@esxi1:~] vmkload_mod -l |grep ixgben

      [root@esxi1:~] esxcfg-nics -l

      Name    PCI          Driver      Link Speed      Duplex MAC Address       MTU    Description

      vmnic0  0000:00:1f.6 ne1000      Up   1000Mbps   Full   94:c6:91:15:0b:e4 9000   Intel Corporation Ethernet Connection (4) I219-V

      [root@esxi1:~] esxcli software vib install -d "/vmfs/volumes/datastore1/VMW-ESX-

      6.7.0-ixgben-1.7.10-offline_bundle-10105563.zip"

      Installation Result

         Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

         Reboot Required: true

         VIBs Installed: INT_bootbank_ixgben_1.7.10-1OEM.670.0.0.8169922

         VIBs Removed: INT_bootbank_ixgben_1.7.1-1OEM.670.0.0.7535516

         VIBs Skipped:

       

      Still no joy after reboot

       

      [root@esxi1:~] esxcli hardware pci list

      0000:06:00.0

         Address: 0000:06:00.0

         Segment: 0x0000

         Bus: 0x06

         Slot: 0x00

         Function: 0x0

         VMkernel Name:

         Vendor Name: Intel Corporation

         Device Name: 82599EB 10-Gigabit SFI/SFP+ Network Connection

         Configured Owner: VMkernel

         Current Owner: VMkernel

         Vendor ID: 0x8086

         Device ID: 0x10fb

         SubVendor ID: 0x8086

         SubDevice ID: 0x7a11

         Device Class: 0x0200

         Device Class Name: Ethernet controller

         Programming Interface: 0x00

         Revision ID: 0x01

         Interrupt Line: 0xff

         IRQ: 255

         Interrupt Vector: 0x00

         PCI Pin: 0x00

         Spawned Bus: 0x00

         Flags: 0x3219

         Module ID: -1

         Module Name: None

         Chassis: 0

         Physical Slot: 4294967295

         Slot Description:

         Passthru Capable: true

         Parent Device: PCI 0:5:1:0

         Dependent Device: PCI 0:6:0:0

         Reset Method: Function reset

         FPT Sharable: true

       

       

      0000:06:00.1

         Address: 0000:06:00.1

         Segment: 0x0000

         Bus: 0x06

         Slot: 0x00

         Function: 0x1

         VMkernel Name:

         Vendor Name: Intel Corporation

         Device Name: 82599EB 10-Gigabit SFI/SFP+ Network Connection

         Configured Owner: VMkernel

         Current Owner: VMkernel

         Vendor ID: 0x8086

         Device ID: 0x10fb

         SubVendor ID: 0x8086

         SubDevice ID: 0x7a11

         Device Class: 0x0200

         Device Class Name: Ethernet controller

         Programming Interface: 0x00

         Revision ID: 0x01

         Interrupt Line: 0xff

         IRQ: 255

         Interrupt Vector: 0x00

         PCI Pin: 0x01

         Spawned Bus: 0x00

         Flags: 0x3219

         Module ID: -1

         Module Name: None

         Chassis: 0

         Physical Slot: 4294967295

         Slot Description:

         Passthru Capable: true

         Parent Device: PCI 0:5:1:0

         Dependent Device: PCI 0:6:0:1

         Reset Method: Function reset

         FPT Sharable: true

       

      [root@esxi1:~] esxcfg-nics -l

      Name    PCI          Driver      Link Speed      Duplex MAC Address       MTU    Description

      vmnic0  0000:00:1f.6 ne1000      Up   1000Mbps   Full   94:c6:91:15:0b:e4 9000   Intel Corporation Ethernet Connection (4) I219-V

        • 1. Re: Can't add 10GBe physical nic
          TomAtTritium Lurker

          I am using Dell / Intel X520-DA2 Dual Port 10Gbe SFP Network Adapter NIC 942V6 Low Profile together with ESXi 6.7 U3.

          NIC shows as "82599EB 10-Gigabit SFP+ Network Connection" in hardware but are not available as a NIC.

           

          I checked the log..

          cat /var/log/vmkernel.log  | grep ixgben

          and found that the SFP+ that i am trying to use is unsupported and that has stopped the 10GB from initializing properly:

          2019-09-10T20:49:43.630Z cpu51:2098034)ixgben: ixgben_InitSharedCode:185: Unsupported SFP+ or QSFP detected. Try to replace a supported module then reload the driver.

           

          I will try another SFP+ or DAC, and update this post with result.

          • 2. Re: Can't add 10GBe physical nic
            4n3w Lurker

            Hey there,

             

            I have what looks like the same issue as ElliottN with some X520-DA2 Dual Port 10GE NIC (FWIW with correct and genuine Intel SFP+ modules) in an Intel NUC8i5BEH (with latest BIOS firmware) machine connected via a Sonnet Echo Express SEL Thunderbolt 3 PCIe expansion chassis. I know this card works just fine because when I boot into ubuntu I'm able to use the card without error.

             

            I've tried many versions of the ixgben driver, all have yielded the same results (PCI device is detected but esxcfg-nics -l only yields the built in ethernet adapter -- here's a list of the drivers I've tried so far:

             

             

            1.7.1-1OEM.670.0.0.7535516
            1.7.1.16-1vmw.670.3.73.14320388
            1.7.10-1OEM.670.0.0.8169922
            1.7.15-1OEM.670.0.0.8169922
            1.7.17-1OEM.670.0.0.8169922
            1.7.20-1OEM.670.0.0.8169922

             

            Every time I reboot after esxcli software vib install I get only the built-in ethernet adapter, and not the ixgben driven adapter.

             

            Also, when I run vmkload_mod -l | grep ixgben I've never seen the ixgben driver get loaded, although I do see the ixgben driver eg:

             

            esxcli software vib list | grep ixgben

            ixgben                         1.7.10-1OEM.670.0.0.8169922           INT     VMwareCertified   2019-09-14

             

             

            Also, here's the evidence the PCI devices are detected.

             

            lspci -v | grep -A1 -i ethernet

            0000:00:1f.6 Network controller Ethernet controller: Intel Corporation Ethernet Connection (6) I219-V [vmnic0]

              Class 0200: 8086:15be

            --

            0000:07:00.0 Network controller Ethernet controller: Intel(R) 82599 10 Gigabit Dual Port Network Connection

              Class 0200: 8086:10fb

            --

            0000:07:00.1 Network controller Ethernet controller: Intel(R) 82599 10 Gigabit Dual Port Network Connection

              Class 0200: 8086:10fb

             

            I'm relatively new to ESXi/vsphere but I've learned a lot troubleshooting this. Curious if there's something I'm missing to add the card? I don't see any errors like TomAtTritium does regarding the SFP+ modules. Any help would be appreciated.

             

            Failing that, maybe a recommendation for a card that will work without this much hassle.

             

            Thanks,

            Andrew

            • 3. Re: Can't add 10GBe physical nic
              ashishsingh1508 Enthusiast
              vExpert

              Most probably an unsupported SFP

               

              OR

               

              FW is not compatible