BUG: Passthrough HBA Controller with VMWare 7.0

[ERMGermany]

System: Dell R730 with Dell HBA330 Controller (LSI 3008) on VMWare 7.0b Build16324942.

Goal: Passthrough Controller to VM

Issue: Passthrough not possible with disks attached to HBA330

Steps:

1. Boot ESXI 7.0 with HBA330 Controller without disks attached

2. vSphere client:

• select host
• Configure->Hardware->PCI-Devices
• "CONFIGURE PASSTHROUGH"
• select HBA330 controller
• HBA330 is listed in "Passtrough-enabled devices"

3. vShpere client

• select VM
• Context menu: Edit Settings
• "ADD NEW DEVICE"
• PCI Device
• select Passtrough device HBA330
• boot VM

3. VM (e.g. freeBSD)

• camcontrol devlist: controller shown
• attach disks to controller (hot swap bay): disks recognized and accessable:

BUG:

After rebooting HOST (with disks attached) the PCI device is not passthrough anymore:

VM-Settings:

Host->PCI-Devices:

No Passthrough possible with attached disks:

Rebooting host does not help. Only detach of disks on boot enables the passthrough again.

[khiregange]

Probable cause - > Changes are not persistent across reboot looks like Esxi is running on stateless mode or read only

can you run the command - >"esxcli hardware pci list "  and share the information which is highlighted.

Address: 0000:0b:00.0

   Segment: 0x0000

   Bus: 0x0b

   Slot: 0x00

   Function: 0x0

   VMkernel Name: vmhba1

   Vendor Name: Avago (LSI)

   Device Name: MegaRAID SAS Invader Controller

   Configured Owner: VM Passthru

   Current Owner: VM Passthru

   Vendor ID: 0x1000

   Device ID: 0x005d

   SubVendor ID: 0x1014

   SubDevice ID: 0x0454

   Device Class: 0x0104

   Device Class Name: RAID bus controller

   Programming Interface: 0x00

   Revision ID: 0x02

   Interrupt Line: 0x0b

   IRQ: 255

   Interrupt Vector: 0x00

   PCI Pin: 0x00

   Spawned Bus: 0x00

   Flags: 0x3401

   Module ID: 23

   Module Name: pciPassthru

   Chassis: 0

   Physical Slot: 9

   Slot Description: Slot 9

   Passthru Capable: true

   Reset Method: Bridge reset

   FPT Sharable: true

Share the above output before rebooting (enabling the PCI passthrough) and after the reboot of the host

[ERMGermany]

I paste three outputs:

1. direct after configuring passthrough

2. after esxi reboot (without disks)

3. after esxi reboot (with disks)

Between step 1 und step 2 following changes:

   Configured Owner: VM Passthru

   Current Owner: VM Passthru

becomes:

   Configured Owner: VMkernel  :smileyalert:

   Current Owner: VMkernel    :smileyalert:

   Module Name: pciPassthru

becomes:

   Module Name: lsi_msgpt3   :smileyalert:

Between step 2 and step 3 following changes:

=> No Change

1. With new passtrough activation (without disks):

0000:02:00.0

   Address: 0000:02:00.0

   Segment: 0x0000

   Bus: 0x02

   Slot: 0x00

   Function: 0x0

   VMkernel Name: vmhba0

   Vendor Name: Avago (LSI Logic)

   Device Name: Dell HBA330 Mini

   Configured Owner: VM Passthru

   Current Owner: VM Passthru

   Vendor ID: 0x1000

   Device ID: 0x0097

   SubVendor ID: 0x1028

   SubDevice ID: 0x1f53

   Device Class: 0x0107

   Device Class Name: Serial Attached SCSI controller

   Programming Interface: 0x00

   Revision ID: 0x02

   Interrupt Line: 0xff

   IRQ: 255

   Interrupt Vector: 0x00

   PCI Pin: 0x00

   Spawned Bus: 0x00

   Flags: 0x3001

   Module ID: 34

   Module Name: pciPassthru

   Chassis: 0

   Physical Slot: -1

   Slot Description: Integrated RAID

   Device Layer Bus Address: m00008a01

   Passthru Capable: true

   Parent Device: PCI 0:0:1:0

   Dependent Device: PCI 0:2:0:0

   Reset Method: Function reset

   FPT Sharable: true

2. After reboot (without disks:

0000:02:00.0

   Address: 0000:02:00.0

   Segment: 0x0000

   Bus: 0x02

   Slot: 0x00

   Function: 0x0

   VMkernel Name: vmhba0

   Vendor Name: Avago (LSI Logic)

   Device Name: Dell HBA330 Mini

   Configured Owner: VMkernel    :smileyalert:

   Current Owner: VMkernel    :smileyalert:

   Vendor ID: 0x1000

   Device ID: 0x0097

   SubVendor ID: 0x1028

   SubDevice ID: 0x1f53

   Device Class: 0x0107

   Device Class Name: Serial Attached SCSI controller

   Programming Interface: 0x00

   Revision ID: 0x02

   Interrupt Line: 0xff

   IRQ: 255

   Interrupt Vector: 0x00

   PCI Pin: 0x00

   Spawned Bus: 0x00

   Flags: 0x3001

   Module ID: 26

   Module Name: lsi_msgpt3     :smileyalert:

   Chassis: 0

   Physical Slot: -1

   Slot Description: Integrated RAID

   Device Layer Bus Address: m00008a01

   Passthru Capable: true

   Parent Device: PCI 0:0:1:0

   Dependent Device: PCI 0:2:0:0

   Reset Method: Function reset

   FPT Sharable: true

3. After reboot WITH disks:

0000:02:00.0

   Address: 0000:02:00.0

   Segment: 0x0000

   Bus: 0x02

   Slot: 0x00

   Function: 0x0

   VMkernel Name: vmhba0

   Vendor Name: Avago (LSI Logic)

   Device Name: Dell HBA330 Mini

   Configured Owner: VMkernel

   Current Owner: VMkernel

   Vendor ID: 0x1000

   Device ID: 0x0097

   SubVendor ID: 0x1028

   SubDevice ID: 0x1f53

   Device Class: 0x0107

   Device Class Name: Serial Attached SCSI controller

   Programming Interface: 0x00

   Revision ID: 0x02

   Interrupt Line: 0xff

   IRQ: 255

   Interrupt Vector: 0x00

   PCI Pin: 0x00

   Spawned Bus: 0x00

   Flags: 0x3001

   Module ID: 29

   Module Name: lsi_msgpt3

   Chassis: 0

   Physical Slot: -1

   Slot Description: Integrated RAID

   Device Layer Bus Address: m00008a01

   Passthru Capable: true

   Parent Device: PCI 0:0:1:0

   Dependent Device: PCI 0:2:0:0

   Reset Method: Function reset

   FPT Sharable: true

[khiregange]

esxi looks to be not saving / updating the changes across the reboot

How is the esxi booting is it from DAS or SD card or from readonly image?

in order to check if the storage is persistent you may create a file /opt/test.txt and reboot the host to check if the file is still present.

[ERMGermany]

YES, you are right!  The test file is not there anymore after reboot.

But that makes no sence to me: I can configure the esxi (IP-Adresses, VLAN, logins, password, can add datastores, VMs etc.)

I installed esxi on USB-stick to test the passtrough. (Normally we install ESXI on mirrored HDs). Have done that USB-stick-installation several times with several linux-systems for several tests on this server.

I do not understand the point, where I get the readonly to this installation. Any idea what I have to change?

[khiregange]

Could you run the this command after the HBA passthrough is enabled with disk added to VM

/sbin/autobackup.sh

run this before the reboot and check if that preserves the settings

if the above fixes the issue then you can create a cronjob to run autobackup.sh every hour so that any changes made would be persistent upon the reboot.

[ERMGermany]

NO, indeed the esxi host looses this information even with using "/bin/auto-backup.sh"