VMware Cloud Community
djanne
Contributor
Contributor

Forced VUM driver updates causing vSAN to go incompliant

Hi,

So I'm in the process of patching several vSAN clusters from ESXi 6.5 U1 to U2.

For some reason VUM applies a new controller driver along with U2, which is NOT compliant with the vSAN HCL.

This means I have to logon to each server and run some esxcli commands to uninstall the new driver and reinstall the old one.

Very time comsuming and annoying.

Anyone have any suggestions on how to stop VUM from installing a non compliant controller driver?

I've of course tried deselecting it, but I suspect its deeply tied into U2.

The driver in question is lsi_msgpt3.

0 Kudos
5 Replies
TheBobkin
Champion
Champion

Hello djanne ,

What controller is in use?

Do you have vSAN Build Recommendation Engine configured and logged in?

Are you using a vSAN-specific baseline?

vSAN 6.6.1 with VUM Integration - Virtual Blocks

https://storagehub.vmware.com/export_to_pdf/vsan-6-6-1-upgrade-considerations-1

Bob

0 Kudos
djanne
Contributor
Contributor

Hi Bob,

Thanks for replying.

Yes the Build Recommendation Engine is running, and it was the one suggesting an upgrade to 6.5 U2.

From what I gather it will suggest upgrades if needed for vSAN as well as controller drivers, and in the future even firmware.

It will however not "block" updates that cause incompability with the HCL.

So in this case, as lsi-msgpt3 version 16 is included with U2, then it applies that. Unfortunately only version 14 is in the vSAN HCL, so after patching its no longer compliant.

My workaround is, as said, to patch and then uninstall version 16 and reinstall 14. But that is a PITA... I was hoping VMware had a better solution for this.

0 Kudos
TheBobkin
Champion
Champion

Hello djanne

What controller are you referring to here?

Are you using HBA330/HBA330 Mini here or something less common?

If HBA330/HBA330 Mini then you will find that "version 16" (e.g. lsi_msgpt3 version 16.00.01.00) is on the HCL but it requires different Firmware than 14.x.x.x drivers so this is the likely reason it is stating non-compliant in the Health check if you are not updating the Firmware also ( 15.17.09.06).

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio&productid=40063

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio&productid=41261

If you are NOT using HBA330/HBA330 Mini then please ignore the above (and inform us of which controller you are referring :smileygrin: ).

Have you perchance opened a Support Request for more information from our VUM engineers?

Bob

0 Kudos
djanne
Contributor
Contributor

We use Cisco UCS, so its Ciscos adapters.

In this case its the "Cisco 12G Modular SAS Pass through Controller", which we use in multiple production clusters.

VMware Compatibility Guide - I/O Device Search

We also have the RAID version(UCSB-MRAID12G-HE) in a (blade based) cluster in our lab, and it was the same issue with that.

It uses the lsi_mr3 driver, and the latest compatible with vSAN is 7.701. VUM however applied 7.702 when we applied 6.5 U2...

VMware Compatibility Guide - I/O Device Search

We have not opened an SR for this, I suspect this is more of a lacking feature than a bug.

0 Kudos
djanne
Contributor
Contributor

FWIW I hacked together a little script that downgrades the VIB for me.

# Add the VMware Modules

if ( !(Get-Module -Name VMware.VimAutomation.Core -ErrorAction SilentlyContinue) ) {

. “C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1”

. “C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1”

}

# Variables

# localpath cannot be a remote disk(such as SMB).

# TempDatastoreName should be a disk on the ESX server

[String]$ClusterName = "vsan-cl-01"

#[String]$LocalPath = "d:\temp\VMW-ESX-6.5.0-lsi_msgpt3-14.00.04.00-offline_bundle-5292647.zip"

#[String]$DepotName = "VMW-ESX-6.5.0-lsi_msgpt3-14.00.04.00-offline_bundle-5292647.zip"

[String]$LocalPath = "d:\temp\VMW-ESX-6.5.0-lsi_mr3-7.701.17.00-offline_bundle-6546028.zip"

[String]$DepotName = "VMW-ESX-6.5.0-lsi_mr3-7.701.17.00-offline_bundle-6546028.zip"

[String]$TempDatastoreName = "datastore*"

[String]$remVIB = "lsi-mr3"

$VMhosts = Get-Cluster -Name $ClusterName | Get-VMHost

foreach ($VMhost in $VMhosts) {

    # Enter Maintenance Mode

    write-host "Entering Maintenance mode on $VMhost"

    Set-VMhost (get-vmhost $VMhost) -State maintenance -Evacuate -VsanDataMigrationMode EnsureAccessibility | Out-Null

    # Remove old VIBs

    write-host "Removing old VIB"

    $esxcli2 = Get-ESXCLI -VMHost $VMhost -V2   

    $esxcli2.software.vib.remove.Invoke(@{"vibname" = $remVIB})

    # Upload VIB

    write-host "Uploading new VIB"

    $Datastore =  $VMhost | Get-Datastore -Name $TempDatastoreName

    $DatastoreDriveName = "HostStore_" + $VMhost.Name.Split(".")[0]

    $Datastore | New-DatastoreDrive -Name $DatastoreDriveName | Out-Null

    Copy-DatastoreItem -Item $LocalPath -Destination $($DatastoreDriveName + ":\") -Force:$true -Confirm:$false

    Remove-PSDrive -Name $DatastoreDriveName

    # Install new VIB

    write-host "Installing new VIB"

    $HostPath = $Datastore.ExtensionData.Info.Url.remove(0,5) + $DepotName

    $CreateArgs = $esxcli2.software.vib.install.CreateArgs()

    $CreateArgs.depot = $HostPath

    $InstallResponse = $esxcli2.software.vib.install.Invoke($CreateArgs)

    # Restart Host if needed

    if ($InstallResponse.RebootRequired -eq $true) {

        Write-Host "Rebooting '$($VMHost.Name)'..."

        Write-Host "VIBs Installed:"

        $InstallResponse.VIBsInstalled

        $VMhost | Restart-VMHost -Confirm:$false | Out-Null   

        # Wait for Server to show as down

        do {

        sleep 60

        $ServerState = (get-vmhost $VMHost).ConnectionState

        }

        while ($ServerState -ne "NotResponding")

        Write-Host "$VMHost is Down"

        # Wait for server to reboot

        do {

        sleep 90

        $ServerState = (get-vmhost $VMHost).ConnectionState

        Write-Host "Waiting for Reboot …"

        }

        while ($ServerState -ne "Maintenance")

        Write-Host "$VMHost is back up"

        # Exit Maintenance Mode

        write-host "Exiting Maintenance mode on $VMhost"

        Set-VMhost (get-vmhost $VMhost) -State Connected | Out-Null

        Write-Host "Pausing 5 minutes for DRS"

        sleep 300

    }

    else {

        Write-Host "No Reboot for '$($VMHost.Name)' required..."   

    }

}

0 Kudos