6 Replies Latest reply on Sep 22, 2020 7:40 AM by ERMGermany

    BUG: Passthrough HBA Controller with VMWare 7.0

    ERMGermany Lurker

      System: Dell R730 with Dell HBA330 Controller (LSI 3008) on VMWare 7.0b Build16324942.

      Goal: Passthrough Controller to VM

      Issue: Passthrough not possible with disks attached to HBA330

       

      Steps:

      1. Boot ESXI 7.0 with HBA330 Controller without disks attached

      2. vSphere client:

      • select host
      • Configure->Hardware->PCI-Devices
      • "CONFIGURE PASSTHROUGH"
      • select HBA330 controller
      • HBA330 is listed in "Passtrough-enabled devices"

      3. vShpere client

      • select VM
      • Context menu: Edit Settings
      • "ADD NEW DEVICE"
      • PCI Device
      • select Passtrough device HBA330
      • boot VM

      3. VM (e.g. freeBSD)

      • camcontrol devlist: controller shown
      • attach disks to controller (hot swap bay): disks recognized and accessable:

      HBA330 - Passthrough Disks.jpg

      HBA330 - Passthrough Disks - camcontrol.jpg

      BUG:

      After rebooting HOST (with disks attached) the PCI device is not passthrough anymore:

      VM-Settings:

      HBA330 - no Passthrough after reboot - VM.jpg

      Host->PCI-Devices:

      HBA330 - no Passthrough after reboot - Host.jpg

      No Passthrough possible with attached disks:

      HBA330 - no Passthrough with disks attached - Host.jpg

      Rebooting host does not help. Only detach of disks on boot enables the passthrough again.

        • 1. Re: BUG: Passthrough HBA Controller with VMWare 7.0
          khiregange Enthusiast

          Probable cause - > Changes are not persistent across reboot looks like Esxi is running on stateless mode or read only

           

          can you run the command - >"esxcli hardware pci list "  and share the information which is highlighted.

           

          Address: 0000:0b:00.0

             Segment: 0x0000

             Bus: 0x0b

             Slot: 0x00

             Function: 0x0

             VMkernel Name: vmhba1

             Vendor Name: Avago (LSI)

             Device Name: MegaRAID SAS Invader Controller

             Configured Owner: VM Passthru

             Current Owner: VM Passthru

             Vendor ID: 0x1000

             Device ID: 0x005d

             SubVendor ID: 0x1014

             SubDevice ID: 0x0454

             Device Class: 0x0104

             Device Class Name: RAID bus controller

             Programming Interface: 0x00

             Revision ID: 0x02

             Interrupt Line: 0x0b

             IRQ: 255

             Interrupt Vector: 0x00

             PCI Pin: 0x00

             Spawned Bus: 0x00

             Flags: 0x3401

             Module ID: 23

             Module Name: pciPassthru

             Chassis: 0

             Physical Slot: 9

             Slot Description: Slot 9

             Passthru Capable: true

             Reset Method: Bridge reset

             FPT Sharable: true

           

          Share the above output before rebooting (enabling the PCI passthrough) and after the reboot of the host

           

           

          1 person found this helpful
          • 2. Re: BUG: Passthrough HBA Controller with VMWare 7.0
            ERMGermany Lurker

            I paste three outputs:

             

            1. direct after configuring passthrough

            2. after esxi reboot (without disks)

            3. after esxi reboot (with disks)

             

            Between step 1 und step 2 following changes:

               Configured Owner: VM Passthru

               Current Owner: VM Passthru

            becomes:

               Configured Owner: VMkernel 

               Current Owner: VMkernel   

             

               Module Name: pciPassthru

            becomes:

               Module Name: lsi_msgpt3  

             

            Between step 2 and step 3 following changes:

            => No Change

             

            1. With new passtrough activation (without disks):

             

            0000:02:00.0

               Address: 0000:02:00.0

               Segment: 0x0000

               Bus: 0x02

               Slot: 0x00

               Function: 0x0

               VMkernel Name: vmhba0

               Vendor Name: Avago (LSI Logic)

               Device Name: Dell HBA330 Mini

               Configured Owner: VM Passthru

               Current Owner: VM Passthru

               Vendor ID: 0x1000

               Device ID: 0x0097

               SubVendor ID: 0x1028

               SubDevice ID: 0x1f53

               Device Class: 0x0107

               Device Class Name: Serial Attached SCSI controller

               Programming Interface: 0x00

               Revision ID: 0x02

               Interrupt Line: 0xff

               IRQ: 255

               Interrupt Vector: 0x00

               PCI Pin: 0x00

               Spawned Bus: 0x00

               Flags: 0x3001

               Module ID: 34

               Module Name: pciPassthru

               Chassis: 0

               Physical Slot: -1

               Slot Description: Integrated RAID

               Device Layer Bus Address: m00008a01

               Passthru Capable: true

               Parent Device: PCI 0:0:1:0

               Dependent Device: PCI 0:2:0:0

               Reset Method: Function reset

               FPT Sharable: true

             

             

             

            2. After reboot (without disks:

             

            0000:02:00.0

               Address: 0000:02:00.0

               Segment: 0x0000

               Bus: 0x02

               Slot: 0x00

               Function: 0x0

               VMkernel Name: vmhba0

               Vendor Name: Avago (LSI Logic)

               Device Name: Dell HBA330 Mini

               Configured Owner: VMkernel   

               Current Owner: VMkernel   

               Vendor ID: 0x1000

               Device ID: 0x0097

               SubVendor ID: 0x1028

               SubDevice ID: 0x1f53

               Device Class: 0x0107

               Device Class Name: Serial Attached SCSI controller

               Programming Interface: 0x00

               Revision ID: 0x02

               Interrupt Line: 0xff

               IRQ: 255

               Interrupt Vector: 0x00

               PCI Pin: 0x00

               Spawned Bus: 0x00

               Flags: 0x3001

               Module ID: 26

               Module Name: lsi_msgpt3    

               Chassis: 0

               Physical Slot: -1

               Slot Description: Integrated RAID

               Device Layer Bus Address: m00008a01

               Passthru Capable: true

               Parent Device: PCI 0:0:1:0

               Dependent Device: PCI 0:2:0:0

               Reset Method: Function reset

               FPT Sharable: true

             

            3. After reboot WITH disks:

             

            0000:02:00.0

               Address: 0000:02:00.0

               Segment: 0x0000

               Bus: 0x02

               Slot: 0x00

               Function: 0x0

               VMkernel Name: vmhba0

               Vendor Name: Avago (LSI Logic)

               Device Name: Dell HBA330 Mini

               Configured Owner: VMkernel

               Current Owner: VMkernel

               Vendor ID: 0x1000

               Device ID: 0x0097

               SubVendor ID: 0x1028

               SubDevice ID: 0x1f53

               Device Class: 0x0107

               Device Class Name: Serial Attached SCSI controller

               Programming Interface: 0x00

               Revision ID: 0x02

               Interrupt Line: 0xff

               IRQ: 255

               Interrupt Vector: 0x00

               PCI Pin: 0x00

               Spawned Bus: 0x00

               Flags: 0x3001

               Module ID: 29

               Module Name: lsi_msgpt3

               Chassis: 0

               Physical Slot: -1

               Slot Description: Integrated RAID

               Device Layer Bus Address: m00008a01

               Passthru Capable: true

               Parent Device: PCI 0:0:1:0

               Dependent Device: PCI 0:2:0:0

               Reset Method: Function reset

               FPT Sharable: true

            • 3. Re: BUG: Passthrough HBA Controller with VMWare 7.0
              khiregange Enthusiast

              esxi looks to be not saving / updating the changes across the reboot

               

              How is the esxi booting is it from DAS or SD card or from readonly image?

               

              in order to check if the storage is persistent you may create a file /opt/test.txt and reboot the host to check if the file is still present.

              • 4. Re: BUG: Passthrough HBA Controller with VMWare 7.0
                ERMGermany Lurker

                YES, you are right!  The test file is not there anymore after reboot.

                 

                But that makes no sence to me: I can configure the esxi (IP-Adresses, VLAN, logins, password, can add datastores, VMs etc.)

                I installed esxi on USB-stick to test the passtrough. (Normally we install ESXI on mirrored HDs). Have done that USB-stick-installation several times with several linux-systems for several tests on this server.

                 

                I do not understand the point, where I get the readonly to this installation. Any idea what I have to change?

                • 5. Re: BUG: Passthrough HBA Controller with VMWare 7.0
                  khiregange Enthusiast

                  Could you run the this command after the HBA passthrough is enabled with disk added to VM

                   

                  /sbin/autobackup.sh

                   

                  run this before the reboot and check if that preserves the settings

                   

                  if the above fixes the issue then you can create a cronjob to run autobackup.sh every hour so that any changes made would be persistent upon the reboot.

                  • 6. Re: BUG: Passthrough HBA Controller with VMWare 7.0
                    ERMGermany Lurker

                    NO, indeed the esxi host looses this information even with using "/bin/auto-backup.sh"