VMware Cloud Community
DCurrie
Contributor
Contributor
Jump to solution

Using Update Manager to upgrade ESX 5.1 to 5.5 - ERROR Conflicting VIBS


Hi

I hope somebody can help. I have a number of 5.1 ESX hosts which I would like to upgrade to 5.5 using Update Manager which seems straight forward enough....... right

vCentre is at 5.5

Import the ESXi ISO to Update Manager

Create a Baseline

Attach Baseline to the host to be upgraded

Scan and Remediate

.... Simple

The problem I am having is that we use the HP Custom image as they are all HP servers, makes sense, right. I have downloaded the HP 5.5 Custom Image ISO from the VMWare site and imported the ISO to Update Manager, created a baseline, attached it to a test host ( DL585 G5 ) and run a scan. Im then presented with an incompatible warning and the details show the below

Conflicting VIBS error.JPG

I have been searching the net for a solution and found multiple entries about dealing with conflicting VIBs and creating custom ISOs, which I have also downloaded the HP ESX Depot ZIP file and created a custom ISO which does not work either.

Below is the current Broadcom VIBs installed on my ESX test host

Broadcom drivers Installed.JPG

I also cant remove the conflicting VIBs as they are the network card drivers and if I do remove them and reboot the host has not network adapters ( yup, i tried that ).

Its been 2 days of pulling my hair out :smileyangry: and maybe I am missing something simple.

Anybody have any ideas ?

:smileyconfused:

Thanks

1 Solution

Accepted Solutions
kwohlgemuth
Contributor
Contributor
Jump to solution

DCurrie,

I am in the same boat as you. I have all HP hosts DL380s. I migrated my test hosts to my vCenter 5.5 U1 server, and went to perform the upgrade on the hosts from 5.1. I first had to apply the latest patches on 5.1, which then allowed me to upgrade the host to 5.5.0. I then had another series of patches that I applied (assuming I had to do that before upgrading to 5.5.1). After applying those, I still have 5.5.1 as an incompatible upgrade.

The upgrade contains the following set of conflicting VIBs:

Broadcom_bootbank_net-bnx2_2.2.5dv55.2-1OEM.550.0.0.1331820

Remove the conflicting VIBs or use Image Builder.....

So, I was at least able to make it to 5.5.0, but no further. Both the 5.5.0 and 5.5.1 ISOs used are the HP OEM ISOs.

I have bnx2_2.2.4f.v55.3 currently installed.

HP has bnx2_2.2.3t.v55.7 as downloadable under the vSphere 5.5 section.

VMware under the HCL has multiple versions including my current version and the bnx2_2.2.5d.v55.2.

Since this is test, I will play around with it some more and see what I can figure out.

----------------------------------------

Update: From Release Notes of 5.5 U1b

Update Manager reports the compliance status as Incompatible when scanning or remediating ESXi 5.x hosts that belong to an HA cluster 

When you perform an upgrade scan of ESXi 5.x hosts that belong to an HA cluster, Update Manager might report the compliance status of the hosts as Incompatible and might not allow the hosts to be remediated. The Incompatible compliance status is because of the way the FDM (HA) agent is installed on ESXi 5.x hosts. Starting with vSphere 5.0, the FDM agent is installed on ESXi hosts as a VIB. When a VIB is installed or updated on an ESXi host, a flag is set to signify that the bootbank on the host has been updated. Update Manager checks for this flag while performing an upgrade scan or remediation and requires this flag to be cleared before upgrading a host. The flag can be cleared by rebooting the host. 

Workaround: Reboot hosts that report the compliance status as Incompatible. Run the upgrade scan or remediation again after the host is back online.

Going to give this a try and I will let you know how it goes.

View solution in original post

36 Replies
kwohlgemuth
Contributor
Contributor
Jump to solution

DCurrie,

I am in the same boat as you. I have all HP hosts DL380s. I migrated my test hosts to my vCenter 5.5 U1 server, and went to perform the upgrade on the hosts from 5.1. I first had to apply the latest patches on 5.1, which then allowed me to upgrade the host to 5.5.0. I then had another series of patches that I applied (assuming I had to do that before upgrading to 5.5.1). After applying those, I still have 5.5.1 as an incompatible upgrade.

The upgrade contains the following set of conflicting VIBs:

Broadcom_bootbank_net-bnx2_2.2.5dv55.2-1OEM.550.0.0.1331820

Remove the conflicting VIBs or use Image Builder.....

So, I was at least able to make it to 5.5.0, but no further. Both the 5.5.0 and 5.5.1 ISOs used are the HP OEM ISOs.

I have bnx2_2.2.4f.v55.3 currently installed.

HP has bnx2_2.2.3t.v55.7 as downloadable under the vSphere 5.5 section.

VMware under the HCL has multiple versions including my current version and the bnx2_2.2.5d.v55.2.

Since this is test, I will play around with it some more and see what I can figure out.

----------------------------------------

Update: From Release Notes of 5.5 U1b

Update Manager reports the compliance status as Incompatible when scanning or remediating ESXi 5.x hosts that belong to an HA cluster 

When you perform an upgrade scan of ESXi 5.x hosts that belong to an HA cluster, Update Manager might report the compliance status of the hosts as Incompatible and might not allow the hosts to be remediated. The Incompatible compliance status is because of the way the FDM (HA) agent is installed on ESXi 5.x hosts. Starting with vSphere 5.0, the FDM agent is installed on ESXi hosts as a VIB. When a VIB is installed or updated on an ESXi host, a flag is set to signify that the bootbank on the host has been updated. Update Manager checks for this flag while performing an upgrade scan or remediation and requires this flag to be cleared before upgrading a host. The flag can be cleared by rebooting the host. 

Workaround: Reboot hosts that report the compliance status as Incompatible. Run the upgrade scan or remediation again after the host is back online.

Going to give this a try and I will let you know how it goes.

DCurrie
Contributor
Contributor
Jump to solution

Hi Kwohlgemuth

Firstly thanks for replying. An update on where Im at

On reading your post I came intot the office this morning and the first thing I did was download the HP 5.5.0 GA ISO to use instead of the HP 5.5u1 ISO. When I use this one the upgrade happens without any errors or compatibility issues. After pushing out 5.5GA, there were a number of critical and no-critical patches to install, one of which was Update 1..... test host is now at 5.5 Update 1 with all patches installed.

Im not sure who updates the custom ISOs whether its VMWare or HP but looks like somebody messed up with this driver in the HP 5.5u1 custom ISO.

Thanks

kwohlgemuth
Contributor
Contributor
Jump to solution

DCurrie,

Thanks for the update, and I would agree that something is a little off maybe with the HP OEM 5.5u1 ISO. Maybe its good for building a new host from scratch rather than upgrading? Also, due to sleep/coffee deprivation (at least that is what I am going to go with), my systems were up to date as well after running the patches post the 5.5.0 upgrade. I should have checked my build numbers more closely. Glad we were able to work through this together!

Regards,

KWohlgemuth

Reply
0 Kudos
ianc1990
Enthusiast
Enthusiast
Jump to solution

Hi Guys,

I'm having the exact same issues here.  I don't actually have update manager - had never heard of it before this post - is it included with enterprise and where would I get it from?

As I don't have update manager, I need to update to 5.5.0 U1 manually(from 5.5.0) but booting the host from the ISO, then selecting to update.  I use the HP custom image to do this on our GEN8 server and I get the error as above(or very similar)

VM1.JPG

Has anyone managed to find a workaround for this?  When I list the VIBs on the host, there isn't one that matches the error so I cannot manually uninstall it.

Thanks

Reply
0 Kudos
kwohlgemuth
Contributor
Contributor
Jump to solution

ianc1990,

Update Manager is part of the vCenter Server install. The documentation can be found here: vSphere Update Manager Documentation. Basically it just requires a small database and a bit of disk for storing the patches and updates. On the documentation page is a sizing calculator to determine how much you will need for your environment. Once it is installed though, it makes updating and upgrading your hosts pretty painless. You just add the plugin to your vCenter client, and you will see a new tab for Update Manager. From there you can scan your hosts to see what patches you need and apply them. You can also upload the ISO or ZIP images for vsphere and use that upgrade your hosts. As for the issue that you are having though, I am not sure. I would have to play around a bit with the image builder first (haven't used it before) to determine what needs to be done to make a custom ISO for installing.

Regards,

KWohlgemuth

Reply
0 Kudos
OlivierLL
Contributor
Contributor
Jump to solution

Hi Guys,

Just to let you know that I also faced the same issue. I wanted to update a HP host from v5.1u2 to v5.5u1 and got the same vibs conflicting error.

I was able to update to v.5.5 but I am getting the same error when trying to apply the v5.5u1 after that.

I am not using Vsphere Update Manager so I have to apply the updates by booting CD/USB.

This is definitively because some Broadcom drivers in v5.5u1 are older than those included in v5.5 and even in v5.1u2...

I will try to report this somewhere to hopefully get a 5.5u2 soon...

Regards

Reply
0 Kudos
whatmyki
Contributor
Contributor
Jump to solution

Hi

I've been hitting the same issue using update manager using the v5.5u1 HP Customised ISO. The method below seems to workaround the issue.

With 5.5 installed, manually INSTALL (not UPDATE, as this fails due to a dependancy) the lastest Broadcom drivers using cli etc.

# esxcli software vib install --depot=/tmp/BCM-NetXtremeII-4.0-offline_bundle-1796156.zip

Once rebooted, try the upgrade to 5.5u1 again. You shouldn't see the validation errors like those described in the original post.

Hope this helps somebody out.

tfontillas99
Contributor
Contributor
Jump to solution

Thank you whatmyki! This has worked for me on G6 and Gen8's but I'm having an issue with G7's.  Not all the NIC's are showing up.  I've tried upgrading the Emulex firmware with no luck.

Any ideas?

Tim F.

Reply
0 Kudos
Robert_BE
Contributor
Contributor
Jump to solution

https://communities.vmware.com/message/2398356?tstart=0#2398356

(reinstall all the Broadcom VIB)

Help but not for Update Manager.

Reply
0 Kudos
JimCFG
Contributor
Contributor
Jump to solution

just got off the phone with HP, here's their solution

  1. Download BCM-NetXtremeII-4.0-1796156.zip from link http://vibsdepot.hp.com/hpq/jun2014/esxi-550-devicedrivers/
  2. Copy the zip file to /tmp folder on ESXi 5.5
  3. Login to Esxi 5.5 server ssh console with root privileges and change directory to tmp.

      /tmp # unzip BCM-NetXtremeII-4.0-1796156.zip

      /tmp # esxcli software vib install --depot=/tmp/BCM-NetXtremeII-4.0-offline_bundle-1796156.zip

  1. Reboot server.
  2. Try to upgrade ESXi 5.5 server to ESXi 5.5 Update01.

mitsumaui
Contributor
Contributor
Jump to solution

Hopefully this is a workaround rather than a solution from HP!

FYI - VMware have published a KB on this: http://kb.vmware.com/kb/2082027

I also have a case open with HP - and so far there seems to be a bit of to and fro as to who / where the fix will be implemented, but the last I heard is that QLogic are going to Rev the driver version to work around this, which then HP need to bundle into their ISO and test / release.

For the time being, I have either resorted to using the March HP ISOs, or removed the VIBs prior to upgrading with the June image.

I'll chase HP today and see where the progression is for a replacement ISO and post back.

TedBWork
Contributor
Contributor
Jump to solution

Any luck on this?  I am up against the same problem and now I have different builds in my server farm

Reply
0 Kudos
JimCFG
Contributor
Contributor
Jump to solution

they wont give you a new ISO.  you just download the updated driver, and build your own ISO using the VMware documentation

Reply
0 Kudos
mitsumaui
Contributor
Contributor
Jump to solution

Nothing yet... I was told by HP TSE he would have update for me in 24 to 48 hours...

I've also got my VMware TAM on the case to see if he can find any further info. Will let you know when I hear more.

Reply
0 Kudos
kwohlgemuth
Contributor
Contributor
Jump to solution

I haven't tried this yet, but VMware Front Experience has a customizer for building/customizing ISOs.

VMware Front Experience: ESXi-Customizer

Reply
0 Kudos
dsohayda
Enthusiast
Enthusiast
Jump to solution

It's nice that that kb article links back to this thread. so much for using the latest/greatest version of install media...

however, when I manually install the offline bundle it reports the following:

Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

   Reboot Required: true

   VIBs Installed: Broadcom_bootbank_scsi-bnx2fc_1.7a.08.v55.1-1OEM.550.0.0.1331820

   VIBs Removed: Broadcom_bootbank_scsi-bnx2fc_1.78.76.v50.1-1OEM.500.0.0.469512

   VIBs Skipped: Broadcom_bootbank_misc-cnic-register_1.7a.02.v55.1-1OEM.550.0.0.1331820, Broadcom_bootbank_net-bnx2_2.2.5d.v55.2-1OEM.550.0.0.1331820,

Broadcom_bootbank_net-bnx2x_1.7a.10.v55.1-1OEM.550.0.0.1331820, Broadcom_bootbank_net-cnic_1.7a.05.v55.3-1OEM.550.0.0.1331820, Broadcom_bootbank_scsi-

bnx2i_2.7a.03.v55.2-1OEM.550.0.0.1331820

seems like it still won't install the VIBs

Reply
0 Kudos
bpjones
Contributor
Contributor
Jump to solution

HP introduced this "feature" a month ago and no fix...  Looks like they're going to leave use to fend for ourselves.  I'm currently on HP's ESXi 5.1.0 build and I've been unable to update to any later 5.1 build or 5.5 builds.

I decided to build a custom ISO.  I'm testing my freshly upgraded host now, but all looks good.  The ESXi-Customizer looks nice but has one critical limitation, the resulting ISO cannot be imported to VUM.  I've not used PowerCLI (necessary to build custom images) since sitting through an advanced session at VMworld a few years ago, but I got through it ok.  Referenced article 2082027 to get the proper nic driver.  After installing PowerCLI, here's the steps I used to build the installer:

Create staging folder, place offline bundles for nic driver and esx installer inside.

1) Add-EsxSoftwareDepot .\VMware-ESXi-5.5.0-Update1-1746018-HP-5.74.27-Jun2014-depot.zip

2) Get-EsxImageProfile   (Looking something like "HP-ESXi-5.5.0-Update1-5.74.27")

3) New-EsxImageProfile -CloneProfile "HP-ESXi-5.5.0-Update1-5.74.27" -Name "HP-ESXi-5.5.0-U1-032014-Respin" -AcceptanceLevel CommunitySupported

    You'll be prompted for Vendor, just type "Hewlett-Packard"

4) Remove-EsxSoftwarePackage -ImageProfile HP-ESXi-5.5.0-U1-032014-Respin -SoftwarePackage misc-cnic-register,net-bnx2,net-bnx2x,net-cnic,scsi-bnx2fc,scsi-bnx2i

5) Export-EsxImageProfile -ImageProfile HP-ESXi-5.5.0-U1-032014-Respin -ExportToBundle -FilePath ./HP-ESXi-5.5.0-U1-032014-Respin.zip

Close and Reopen PowerCLI.  I'm not entirely sure this is necessary, but I was having trouble determining if the packages where being removed and added to the proper profile.  So I saved my cloned profile and reopened to verify it had removed the broadcomm drivers, it had. Re-open the new profile:

6) Add-EsxSoftwareDepot ./HP-ESXi-5.5.0-U1-032014-Respin.zip

7) Add-EsxSoftwareDepot ./BCM-NetXtremeII-3.0-offline_bundle-1566459.zip

😎 Add-EsxSoftwarePackage -ImageProfile HP-ESXi-5.5.0-U1-032014-Respin -SoftwarePackage misc-cnic-register,net-bnx2,net-bnx2x,net-cnic,scsi-bnx2fc,scsi-bnx2i

9) Export-EsxImageProfile -ImageProfile HP-ESXi-5.5.0-U1-032014-Respin -ExportToBundle -FilePath ./HP-ESXi-5.5.0-U1-032014-Respin.Final.zip  (This is just a backup of the bundle for easy modification)

10) Export-EsxImageProfile -imageprofile HP-ESXi-5.5.0-U1-032014-Respin -ExportToIso -FilePath ./HP-ESXi-5.5.0-U1-032014-Respin.Final.iso

Imported the ISO to VUM, created new Baseline and now I'm able to initiate host remediation.  My test host completed successfully and everything looks good.  But I wonder what hoops I'll have to jump through next time. I saw someone mention they had troubles with a G7 host, my test host is a G7.

mitsumaui
Contributor
Contributor
Jump to solution

Nice - I am going to try this in our lab too... But the inability to import this into VUM is a dealbreaker for me...

HP came back to me on Friday and said that they are not planning another HP ISO until their next release cycle (September) - If Custom image import into VUM really does not work, i'm going to be requesting escalation with HP.

Reply
0 Kudos
bpjones
Contributor
Contributor
Jump to solution

The method I outlined will work with VUM, because it uses the powercli tools to build the image.  Someone had referenced  a 3rd party tool which will not produce a VUM compatible image.

Reply
0 Kudos