VMware Cloud Community
SergeyKV
Contributor
Contributor

SR-IOV not Enable

Hello, please help. I am using Vmware ESXi 6.0U2 and network cards Mellanox:

  Device Type:      ConnectX3
  Part Number:      MCX312A-XCB_A2-A6
  Description:      ConnectX-3 EN network interface card; 10GigE; dual-port SFP+; PCIe3.0 x8 8GT/s; RoHS R6
  PSID:             MT_1080120023
  PCI Device Name:  mt4099_pci_cr1
  Port1 MAC:        e41d2de5a400
  Port2 MAC:        e41d2de5a401
  Versions:         Current        Available
     FW             2.35.5100      N/A
     PXE            3.4.0648       N/A

I have installed different drivers and different firmware for ESXi,
but the SR-IOV and is not Enable (01.jpg).
The last that I used these drivers together with this firmware ( FW
2.35.5100 and MEL-mlnx-3.15.2.0-offline_bundle-3660328.zip) :
https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI60-MELLANOX-MLNX-NATIVE-ESX-31520-1O...

I also used the latest driver and firmware, but the SR-IOV and is not
Enable. (02.jpg)
(fw-ConnectX3-rel-2_36_5000-MCX312A-XCB_A2-A6-FlexBoot-3.4.718.bin.zip
and MLNX-NATIVE-ESX-ConnectX-3-3.2.0.15-10EM-600.0.0.2768847.zip).

Please tell me what drivers and firmware used to enable SR-IOV on the ESXi 6.0U2?

Tags (3)
Reply
0 Kudos
7 Replies
SergeyKV
Contributor
Contributor

Error in vpxa.log:

2016-04-11T09:13:03.203Z info vpxa[FFD3DB70] [Originator@6876 sub=Default opID=48c126d3-4a6a-4551-a076-93257984fbe5-15553-ngc-6c-ca] [VpxLRO] -- ERROR task-internal-48 -- pciPassthruSystem -- vim.host.PciPassthruSystem.updatePassthruConfig: vim.fault.PlatformConfigFault:

--> Result:

--> (vim.fault.PlatformConfigFault) {

-->    faultCause = (vmodl.MethodFault) null,

-->    text = "Failed to update SR-IOV configuration of module(s): nmlx4_core",

-->    msg = "Received SOAP response fault from [<cs p:1f35fc50, TCP:localhost:8307>]: updatePassthruConfig

--> An error occurred during host configuration."

--> }

--> Args:

-->

--> Arg config:

--> (vim.host.PciPassthruConfig) [

-->    (vim.host.SriovConfig) {

-->       id = "0000:07:00.0",

-->       passthruEnabled = false,

-->       sriovEnabled = true,

-->       numVirtualFunction = 6

-->    }

--> ]

Reply
0 Kudos
Techie01
Hot Shot
Hot Shot

AFAIK, only two drivers support passthrough ( intel and emulex) . Not aware whether mellanox support this. Even though the card ( pci capability support) , the driver support is also needed for passthrough.

Can you please post the output of   below command

esxcli  hardware pci list |egrep -i 'Vendor Name|Passthru'

Reply
0 Kudos
SergeyKV
Contributor
Contributor

[root@localhost:~] esxcli  hardware pci list |egrep -i 'Vendor Name|Passthru'

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: true

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: true

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: true

   Vendor Name: Hewlett-Packard Company

   Passthru Capable: true

   Vendor Name: Matrox Electronics Systems Ltd.

   Passthru Capable: true

   Vendor Name: Hewlett-Packard Company

   Passthru Capable: true

   Vendor Name: Hewlett-Packard Company

   Passthru Capable: true

   Vendor Name: Hewlett-Packard Company

   Passthru Capable: true

   Vendor Name: Broadcom Corporation

   Passthru Capable: true

   Vendor Name: Broadcom Corporation

   Passthru Capable: true

   Vendor Name: Broadcom Corporation

   Passthru Capable: true

   Vendor Name: Broadcom Corporation

   Passthru Capable: true

   Vendor Name: Mellanox Technologies

   Passthru Capable: true

   Vendor Name: Mellanox Technologies

   Passthru Capable: true

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

   Vendor Name: Intel Corporation

   Passthru Capable: false

[root@localhost:~]

Reply
0 Kudos
Techie01
Hot Shot
Hot Shot

I dont have a clear answer, but the output shows that the device support passthrough. Possibilities are.

Check if mellanox driver supports passthrough

Check what is the output of 'esxcli network sriovnic list'  .

Check if you are able to enable it from VI-client ( and not from webclient)

Check whether you are able to see VF's in vsish ( /hardware/pci/seg/0/bus/36/slot/0/func/1/maxNumVFs) . The numbers change as per your machine

Check if you have enabled SRIOV properly from BIOS and you are not seeing any warning in the server BIOS during the boot

Reply
0 Kudos
SergeyKV
Contributor
Contributor

1) esxcli network sriovnic list  is clear

[root@localhost:~] esxcli network sriovnic list

[root@localhost:~]

2) From the client can not be activated VI- SR-IOV

3) The BIOS enable support for SR-IOV

4) I do not understand how this check ( /hardware/pci/seg/0/bus/36/slot/0/func/1/maxNumVFs)

5) Drivers version support passthrough (MEL-mlnx-3.15.2.0-offline_bundle-3660328)

Reply
0 Kudos
Techie01
Hot Shot
Hot Shot

4) I do not understand how this check ( /hardware/pci/seg/0/bus/36/slot/0/func/1/maxNumVFs)


type vsish . Then you enter into vsish prompt

There you can traverse into these directories. 


It could be that the driver is currently loaded with 0 VF's. In case if you have a test machine, try to reload the driver with VF's


I have a test machine with elxnet driver , so the following steps are followed ( change elxnet with the mellonox driver name)


esxcfg-module -i elxnet |grep -i max_vfs   -- > If this command gives output, try to reload the driver with VF's

vmkload_mod elxnet max_vfs=4,4

Then try to enable passthrough



Reply
0 Kudos
apurba2403
Contributor
Contributor

I am facing the same issue. So how to enable passthrough capability for the Intel adapter?

Reply
0 Kudos