Update manager still shows 6.5 Update 1 complete needed even though host is running 6.5 U1 installed from HPE Custom ISO
There are 3 VMware updates listed including the full update ALL of which are in the U1 release notes... What gives?
ixgben patch from 3/8/17 and ixgben update from 7/26/17
This is the case for two separate customers with completely different gear. If these updates are installed its purple screen and rollback
The servers are HPE DL380 Gen8
This is the response back from HPE.
I have not had a chance to try this. Hopefully I will get time this afternoon.
PCIe NIC FW/Drivers Details:
NIC Model: HP Ethernet 10Gb 2-port 560SFP+ Adapter (Slot 3)
Driver: ixgbe Version: 4.5.2-iov
Firmware Version: 0x80000835, 1.1618.0
As per PSOD footprint, this issue could be related to intelcim-provider installed on server in question.
Intel_bootbank_intelcim-provider_0.5-3.3:
Name: intelcim-provider
Version: 0.5-3.3
Type: bootbank
Vendor: Intel
Action Plan:
>> To isolate, remove intelcim-provider from server in question. Ideally removing intelcim-provider should not affect functionality of 560SFP+ card installed on server.
NOTE: A host reboot might be required for changes to take effect.
Below instructions could be followed to remove intelcim-provider.
>> Ensure to move ESXi host in maintenance or tech support mode before implementing this change.
>> Connect to ESXi host using putty (SSH) application. Please refer below links to enable SSH for putty session.
OR
http://masteringvmware.com/how-to-enable-ssh-on-esxi-6-using-vsphere-web-client/
>> Once connected to ESXi host using putty (SSH), run below command to stop CIM provider service.
/etc/init.d/sfcbd-watchdog stop
/etc/init.d/sfcbd status
>> Further remove intelcim-provider from ESXi host in question.
esxcli software vib remove -n=intelcim-provider
>> Further run below command to start sfcbd again.
/etc/init.d/sfcbd-watchdog start
>> Once reboot host for changes to take effect.
>> Re-connect to ESXi host using putty application & run below command to confirm sfcbd running.
chkconfig sfcbd-watchdog
chkconfig sfcbd
/etc/init.d/sfcbd status
/etc/init.d/sfcbd-watchdog status
>> Run below command to confirm intelcim-provider no longer present.
esxcli software vib list | grep intelcim-provider (if shows no output, this confirms intelcim-provider no longer present)
>> Further take ESXi host out of maintenance or tech support mode and monitor server for 48-72 hours to confirm, if issue re-occurs.
Can you please attach the output of 'esxcli software vib list' from these machines ( machines which are installed with 65U1 HP custom ISO) ?
scsi-mpt2sas 15.10.07.00-1OEM.550.0.0.1331820 Avago VMwareCertified 2017-08-15
bnxtnet 20.6.101.0-1OEM.650.0.0.4598673 BCM VMwareCertified 2017-08-15
net-tg3 3.137l.v60.1-1OEM.600.0.0.2494585 BRCM VMwareCertified 2017-08-15
elx-esx-libelxima.so 11.2.1238.0-03 ELX VMwareCertified 2017-08-15
brcmfcoe 11.2.1213.0-1OEM.650.0.0.4598673 EMU VMwareCertified 2017-08-15
elxiscsi 11.2.1238.0-1OEM.650.0.0.4598673 EMU VMwareCertified 2017-08-15
elxnet 11.2.1149.0-1OEM.650.0.0.4240417 EMU VMwareCertified 2017-08-15
ima-be2iscsi 11.2.1147.5-1OEM.600.0.0.2494585 EMU VMwareCertified 2017-08-15
lpfc 11.2.247.0-1OEM.650.0.0.4598673 EMU VMwareCertified 2017-08-15
amsd 650.11.1.0-24.4240417 HPE PartnerSupported 2017-08-15
bootcfg 6.0.0.02-03.00.5.2494585 HPE PartnerSupported 2017-08-15
conrep 6.0.0.02-00.00.12.2494585 HPE PartnerSupported 2017-08-15
cru 650.6.5.9.8-1.4240417 HPE PartnerSupported 2017-08-15
fc-enablement 650.3.1.0.16-4240417 HPE PartnerSupported 2017-08-15
hponcfg 6.0.0.5.0-1.5.2494585 HPE PartnerSupported 2017-08-15
ilo 650.10.1.0-12.4240417 HPE PartnerSupported 2017-08-15
oem-build 650.U1.10.1.0.14-4240417 HPE PartnerSupported 2017-08-15
scsi-hpdsa 5.5.0.54-1OEM.550.0.0.1331820 HPE PartnerSupported 2017-08-15
scsi-hpsa 6.0.0.124-1OEM.600.0.0.2494585 HPE VMwareCertified 2017-08-15
scsi-hpvsa 5.5.0.102-1OEM.550.0.0.1331820 HPE PartnerSupported 2017-08-15
smx-provider 650.03.12.10.1-4240417 HPE VMwareAccepted 2017-08-15
ssacli 3.10.3.0-6.5.0.4240417 HPE PartnerSupported 2017-08-15
testevent 6.5.0.02-00.00.16.4240417 HPE PartnerSupported 2017-08-15
hpnmi 600.2.3.14-2159203 Hewlett-Packard PartnerSupported 2017-08-15
net-i40e 2.0.6-1OEM.600.0.0.2494585 INT VMwareCertified 2017-08-15
net-igb 5.3.3-1OEM.600.0.0.2494585 INT VMwareCertified 2017-08-15
net-ixgbe 4.5.1-1OEM.600.0.0.2494585 INT VMwareCertified 2017-08-15
intelcim-provider 0.5-3.3 Intel VMwareAccepted 2017-08-15
nmlx4-core 3.15.5.5-1OEM.600.0.0.2768847 MEL VMwareCertified 2017-08-15
nmlx4-en 3.15.5.5-1OEM.600.0.0.2768847 MEL VMwareCertified 2017-08-15
nmlx4-rdma 3.15.5.5-1OEM.600.0.0.2768847 MEL VMwareCertified 2017-08-15
nmlx5-core 4.16.8.8-1OEM.650.0.0.4598673 MEL VMwareCertified 2017-08-15
nmst 4.6.0.101-1OEM.650.0.0.4598673 MEL PartnerSupported 2017-08-15
smartpqi 1.0.0.1060-1OEM.650.0.0.4598673 MSCC VMwareCertified 2017-08-15
nhpsa 2.0.16-1OEM.650.0.0.4598673 Microsemi VMwareCertified 2017-08-15
qedentv 3.0.6.9-1OEM.650.0.0.4598673 QLC VMwareCertified 2017-08-15
ima-qla4xxx 500.2.02.20-1vmw.0.5.021123 QLogic VMwareCertified 2017-08-15
misc-cnic-register 1.713.30.v60.1-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
net-bnx2 2.2.6b.v60.2-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
net-bnx2x 2.713.30.v60.7-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
net-cnic 2.713.30.v60.5-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
net-nx-nic 6.0.643-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
net-qlcnic 6.1.191-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
qlnativefc 2.1.57.0-1OEM.600.0.0.2768847 QLogic VMwareCertified 2017-08-15
scsi-bfa 3.2.6.0-1OEM.550.0.0.1331820 QLogic VMwareCertified 2017-08-15
scsi-bnx2fc 1.713.30.v60.5-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
scsi-bnx2i 2.713.30.v60.3-1OEM.600.0.0.2494585 QLogic VMwareCertified 2017-08-15
scsi-qla4xxx 644.55.36.0-1OEM.550.0.0.1331820 QLogic VMwareCertified 2017-08-15
ata-libata-92 3.00.9.2-16vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-amd 0.3.10-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-atiixp 0.4.6-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-cmd64x 0.2.5-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-hpt3x2n 0.3.4-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-pdc2027x 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-serverworks 0.4.3-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-sil680 0.4.8-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ata-pata-via 0.3.3-2vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
block-cciss 3.6.14-10vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
char-random 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ehci-ehci-hcd 1.0-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
hid-hid 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
i40en 1.3.1-5vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
igbn 0.1.0.0-14vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ipmi-ipmi-devintf 39.1-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ipmi-ipmi-msghandler 39.1-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ipmi-ipmi-si-drv 39.1-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
lsi-mr3 6.910.18.00-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
lsi-msgpt3 12.00.02.00-11vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
misc-drivers 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
mtip32xx-native 3.9.5-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ne1000 0.8.0-16vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
nenic 1.0.0.2-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-cdc-ether 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-e1000 8.0.3.1-5vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-e1000e 3.2.2.1-2vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-enic 2.1.2.38-2vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-fcoe 1.0.29.9.3-7vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-forcedeth 0.61-2vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-libfcoe-92 1.0.24.9.4-8vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-usbnet 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
net-vmxnet3 1.1.3.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ntg3 4.1.2.0-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
nvme 1.2.0.32-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
nvmxnet3 2.0.0.22-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
ohci-usb-ohci 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
pvscsi 0.1-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
qfle3 1.0.2.7-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
qflge 1.1.0.3-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-ahci 3.0-26vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-ata-piix 2.12-10vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-sata-nv 3.5-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-sata-promise 2.12-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-sata-sil24 1.1-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-sata-sil 2.3-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
sata-sata-svw 2.3-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-aacraid 1.1.5.1-9vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-adp94xx 1.0.8.12-6vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-aic79xx 3.1-5vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-fnic 1.5.0.45-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-ips 7.12.05-4vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-iscsi-linux-92 1.0.0.2-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-libfc-92 1.0.40.9.3-5vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-megaraid-mbox 2.20.5.1-6vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-megaraid-sas 6.603.55.00-2vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-megaraid2 2.00.4-9vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-mptsas 4.23.01.00-10vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
scsi-mptspi 4.23.01.00-10vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
shim-iscsi-linux-9-2-1-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-iscsi-linux-9-2-2-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-libata-9-2-1-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-libata-9-2-2-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-libfc-9-2-1-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-libfc-9-2-2-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-libfcoe-9-2-1-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-libfcoe-9-2-2-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-vmklinux-9-2-1-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-vmklinux-9-2-2-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
shim-vmklinux-9-2-3-0 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
uhci-usb-uhci 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
usb-storage-usb-storage 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
usbcore-usb 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
vmkata 0.1-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
vmkplexer-vmkplexer 6.5.0-1.26.5969303 VMW VMwareCertified 2017-08-15
vmkusb 0.1-1vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
vmw-ahci 1.0.0-39vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
xhci-xhci 1.0-3vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-15
cpu-microcode 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
emulex-esx-elxnetcli 11.1.28.0-1.26.5969303 VMware VMwareCertified 2017-08-15
esx-base 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
esx-dvfilter-generic-fastpath 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
esx-tboot 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
esx-ui 1.21.0-5724747 VMware VMwareCertified 2017-08-15
esx-xserver 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
lsu-hp-hpsa-plugin 2.0.0-12vmw.600.0.0.2494585 VMware VMwareCertified 2017-08-15
lsu-lsi-lsi-mr3-plugin 1.0.0-10vmw.650.1.26.5969303 VMware VMwareCertified 2017-08-15
lsu-lsi-lsi-msgpt3-plugin 1.0.0-7vmw.650.1.26.5969303 VMware VMwareCertified 2017-08-15
lsu-lsi-megaraid-sas-plugin 1.0.0-8vmw.650.1.26.5969303 VMware VMwareCertified 2017-08-15
lsu-lsi-mpt2sas-plugin 2.0.0-6vmw.650.1.26.5969303 VMware VMwareCertified 2017-08-15
lsu-lsi-mptsas-plugin 1.0.0-1vmw.600.0.0.2494585 VMware VMwareCertified 2017-08-15
native-misc-drivers 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
rste 2.0.2.0088-4vmw.650.1.26.5969303 VMware VMwareCertified 2017-08-15
scsi-lpfc820 8.2.3.1-129vmw.550.0.0.1331820 VMware VMwareCertified 2017-08-15
vmware-esx-esxcli-nvme-plugin 1.2.0.10-1.26.5969303 VMware VMwareCertified 2017-08-15
vmware-fdm 6.5.0-5973321 VMware VMwareCertified 2017-08-15
vsan 6.5.0-1.26.5912915 VMware VMwareCertified 2017-08-15
vsanhealth 6.5.0-1.26.5912974 VMware VMwareCertified 2017-08-15
tools-light 6.5.0-1.26.5969303 VMware VMwareCertified 2017-08-15
On a VMware released 65U1 ISO, I can see ixgben vib.
[root@xxx:~] esxcli software vib list |grep -i ixgben
ixgben 1.4.1-2vmw.650.1.26.5969303 VMW VMwareCertified 2017-08-11
[root@xxx:~]
But in the vib list which you have posted, I cant see any ixgben vib. It shows only ixgbe vib.
Since that VIB is missing in the HP custom ISO, update manager suggests to install the missing ixgben vib to get complete 65U1 fixes.
We are having the exact same issue and have a ticket open with vm.
They had me run the latest HP support pack and updated the Bios to the latest version.
The next step was to run Update Manager but exclude the ixgben patches.
I only applied the ESXi 6.5 Complete Update 1. After doing this the server still went into a PSOD.
Hopefully I'll hear back today from vm on what the next steps to try.
On another host I used the previous July HPE ISO 6.5.0d without U1 and let update manager install U1. I can confirm this causes a PSOD as well. Something is up with the HPE ISO and U1.
I can also confirm that trying to upgrade with update manager to the latest ISO from the 6.5.0d version it STILL PSODs... fun times
Please post the PSOD screenshot. The original issue in this thread is related to update manager showing additional patches as applicable and the new issue reported is related to hosts getting PSOD.
Hi ,
For 6.5 version don't do all updates , verify the which component you are updating and check the compatibly in HCL
driver , firmware and all then do updated .
Probably this is relevant to Advisory: HPE Network Adapters - Intel Firmware Component May Not Be Listed Under VMware ESXi Operat...
Configured with any of the following network adapters:
The hosts were updated to the latest firmware prior to upgrade. Latest HPE Custom ISO 6.5 w/ U1 was used for install. It's still detecting U1 as being needed after installed.
There aren't two separate issues here. I mentioned the PSOD in the original post.
Same issue with us. We updated the firmware using the HP SPP-2017.07.1 iso.
We have 6 ESXi servers. I have only upgraded one using the custom HP 6.5 update 1 iso. For now I will leave the others at 6.0 update 3 till this issue is resolved.
We have the same issue where VMware Update Manager thinks ESXi 6.5 Complete Update 1 still needs to be applied. Applying this seems to make the server PSOD.
There is also an issue with the scratch log location when using the HP 6.5u1 iso.
Can someone post the screenshot of PSOD?
Here are a few screen shots. The server is already at 6.5 u1. It was upgraded using the HP 6.5u1 iso.
In the first screen shot I ran a scan for patches and you can see that it wants to apply ESXi 6.5 Complete Update 1.
The second screen shot shows just this update being staged and applied. There are also 2 other updates that want to be applied. They are both the igxben vib updates but I am only applying the Complete update 1 for now cause this seems to PSOD the server.
The last screen shot is the PSOD.
Thanks for any help.
My PSOD are identical
igbn and ixgben should be excluded if you are opting to go with U1 till someone hear from HPE or Intel.
ESXi 6.5 Update 1 PSOD on HPE 460c Gen9 after Ixgben driver update – The vGoodie-bag
As seen in the PSOD, the exception is coming from sfcb-intelci , which is an application from intel to monitor hardware status. This application does not seem to be in vmware ISO, but should be coming through HP ISO or the intel async driver. Good to contact HP to get details on the PSOD
Anyone hear anything from VMware or HPE?
I have not. Vmware has told me to not run the updates after upgrading using the HP 6.5u1 iso.
HP had me update net-ixgbe to version 4.5.2 and then wanted me to try again.
I can see net-ixgbe is now at 4.5.2 but I have the same problem. Update manager still thinks it needs to run the updates.
If I run them the server still does a PSOD. I have replied to HP with this info and waiting to hear back.