System: Dell R730 with Dell HBA330 Controller (LSI 3008) on VMWare 7.0b Build16324942.
Goal: Passthrough Controller to VM
Issue: Passthrough not possible with disks attached to HBA330
Steps:
1. Boot ESXI 7.0 with HBA330 Controller without disks attached
2. vSphere client:
3. vShpere client
3. VM (e.g. freeBSD)
BUG:
After rebooting HOST (with disks attached) the PCI device is not passthrough anymore:
VM-Settings:
Host->PCI-Devices:
No Passthrough possible with attached disks:
Rebooting host does not help. Only detach of disks on boot enables the passthrough again.
Probable cause - > Changes are not persistent across reboot looks like Esxi is running on stateless mode or read only
can you run the command - >"esxcli hardware pci list " and share the information which is highlighted.
Address: 0000:0b:00.0
Segment: 0x0000
Bus: 0x0b
Slot: 0x00
Function: 0x0
VMkernel Name: vmhba1
Vendor Name: Avago (LSI)
Device Name: MegaRAID SAS Invader Controller
Configured Owner: VM Passthru
Current Owner: VM Passthru
Vendor ID: 0x1000
Device ID: 0x005d
SubVendor ID: 0x1014
SubDevice ID: 0x0454
Device Class: 0x0104
Device Class Name: RAID bus controller
Programming Interface: 0x00
Revision ID: 0x02
Interrupt Line: 0x0b
IRQ: 255
Interrupt Vector: 0x00
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x3401
Module ID: 23
Module Name: pciPassthru
Chassis: 0
Physical Slot: 9
Slot Description: Slot 9
Passthru Capable: true
Reset Method: Bridge reset
FPT Sharable: true
Share the above output before rebooting (enabling the PCI passthrough) and after the reboot of the host
I paste three outputs:
1. direct after configuring passthrough
2. after esxi reboot (without disks)
3. after esxi reboot (with disks)
Between step 1 und step 2 following changes:
Configured Owner: VM Passthru
Current Owner: VM Passthru
becomes:
Configured Owner: VMkernel :smileyalert:
Current Owner: VMkernel :smileyalert:
Module Name: pciPassthru
becomes:
Module Name: lsi_msgpt3 :smileyalert:
Between step 2 and step 3 following changes:
=> No Change
1. With new passtrough activation (without disks):
0000:02:00.0
Address: 0000:02:00.0
Segment: 0x0000
Bus: 0x02
Slot: 0x00
Function: 0x0
VMkernel Name: vmhba0
Vendor Name: Avago (LSI Logic)
Device Name: Dell HBA330 Mini
Configured Owner: VM Passthru
Current Owner: VM Passthru
Vendor ID: 0x1000
Device ID: 0x0097
SubVendor ID: 0x1028
SubDevice ID: 0x1f53
Device Class: 0x0107
Device Class Name: Serial Attached SCSI controller
Programming Interface: 0x00
Revision ID: 0x02
Interrupt Line: 0xff
IRQ: 255
Interrupt Vector: 0x00
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x3001
Module ID: 34
Module Name: pciPassthru
Chassis: 0
Physical Slot: -1
Slot Description: Integrated RAID
Device Layer Bus Address: m00008a01
Passthru Capable: true
Parent Device: PCI 0:0:1:0
Dependent Device: PCI 0:2:0:0
Reset Method: Function reset
FPT Sharable: true
2. After reboot (without disks:
0000:02:00.0
Address: 0000:02:00.0
Segment: 0x0000
Bus: 0x02
Slot: 0x00
Function: 0x0
VMkernel Name: vmhba0
Vendor Name: Avago (LSI Logic)
Device Name: Dell HBA330 Mini
Configured Owner: VMkernel :smileyalert:
Current Owner: VMkernel :smileyalert:
Vendor ID: 0x1000
Device ID: 0x0097
SubVendor ID: 0x1028
SubDevice ID: 0x1f53
Device Class: 0x0107
Device Class Name: Serial Attached SCSI controller
Programming Interface: 0x00
Revision ID: 0x02
Interrupt Line: 0xff
IRQ: 255
Interrupt Vector: 0x00
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x3001
Module ID: 26
Module Name: lsi_msgpt3 :smileyalert:
Chassis: 0
Physical Slot: -1
Slot Description: Integrated RAID
Device Layer Bus Address: m00008a01
Passthru Capable: true
Parent Device: PCI 0:0:1:0
Dependent Device: PCI 0:2:0:0
Reset Method: Function reset
FPT Sharable: true
3. After reboot WITH disks:
0000:02:00.0
Address: 0000:02:00.0
Segment: 0x0000
Bus: 0x02
Slot: 0x00
Function: 0x0
VMkernel Name: vmhba0
Vendor Name: Avago (LSI Logic)
Device Name: Dell HBA330 Mini
Configured Owner: VMkernel
Current Owner: VMkernel
Vendor ID: 0x1000
Device ID: 0x0097
SubVendor ID: 0x1028
SubDevice ID: 0x1f53
Device Class: 0x0107
Device Class Name: Serial Attached SCSI controller
Programming Interface: 0x00
Revision ID: 0x02
Interrupt Line: 0xff
IRQ: 255
Interrupt Vector: 0x00
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x3001
Module ID: 29
Module Name: lsi_msgpt3
Chassis: 0
Physical Slot: -1
Slot Description: Integrated RAID
Device Layer Bus Address: m00008a01
Passthru Capable: true
Parent Device: PCI 0:0:1:0
Dependent Device: PCI 0:2:0:0
Reset Method: Function reset
FPT Sharable: true
esxi looks to be not saving / updating the changes across the reboot
How is the esxi booting is it from DAS or SD card or from readonly image?
in order to check if the storage is persistent you may create a file /opt/test.txt and reboot the host to check if the file is still present.
YES, you are right! The test file is not there anymore after reboot.
But that makes no sence to me: I can configure the esxi (IP-Adresses, VLAN, logins, password, can add datastores, VMs etc.)
I installed esxi on USB-stick to test the passtrough. (Normally we install ESXI on mirrored HDs). Have done that USB-stick-installation several times with several linux-systems for several tests on this server.
I do not understand the point, where I get the readonly to this installation. Any idea what I have to change?
Could you run the this command after the HBA passthrough is enabled with disk added to VM
/sbin/autobackup.sh
run this before the reboot and check if that preserves the settings
if the above fixes the issue then you can create a cronjob to run autobackup.sh every hour so that any changes made would be persistent upon the reboot.
NO, indeed the esxi host looses this information even with using "/bin/auto-backup.sh"