ChristTheGreat
Contributor
Contributor

vsphere 6.7 LSI raid card, can't get drive status

Jump to solution

Hi!

I have this server setup at home:

SuperMicro X10SLL-F

Intel Xeon E3-1241 v3

32GB 3CC 1600

LSI 8708EM2

2x Velociraptor 10k 300gb RAID 1

4x WD Red 2TB Raid 5

I was running Vsphere 6.0 since a long time, and I decided to go over 6.7. Installation worked fine, I have access to everything, all is working... Except 2 things:

1. I can't get my LSI 8708EM2 (detected as a 1078) to show drive status

2. Can't get my Storage Manager to access disk info or rebuild raid.

When I was on vSphere 6.0, I was able to see my drive status (and veeam one could alert me), and with my WIndows VM, I had LSI MSM and I was able to configure the raid card.

So what I did:

Install SAS driver I was using in version 6 --> not working

Installed the SMIS I had  --> not working

Installed lastest SMIS  --> not working

Disabled the Firewall --> not working

I have no idea what else I can do. I know it's an old card, but I got it for cheap. Any idea how can I get this fixed?

Thanks

67 Replies
AlexJoda
Contributor
Contributor

There are  not many options left at the moment. The available CIM providers for 6.7 are very few and not one of independent RAID big players are left. We stayed at the moment with 6.5U1 which is the last version able to monitor the disks of the hosts with LSI controllers. Without disk monitoring you cannot operate a productive vSphere host in any way! I don’t understand why nobody is taken care of this!

The only option left might be to use Areca controllers because they have an independent ethernet interface to setup and monitor the RAID and they have at least a controller driver for 6.7...

0 Kudos
peetz
Leadership
Leadership

Greetings all,

I'm late to the party, but I want to share with you an alternate method to detect errors reported by LSI MegaRAID adapters.

I also noticed that the LSI CIM provider is no longer working with ESXi 6.7 and looked in the ESXi host log files for other useful information.

In /var/log/vmkernel.log you can find messages like this one

2018-12-16T13:42:23.459Z cpu0:2097613)<6>megasas_hotplug_work[1]: event code 0x0023

You can look up the meaning of the event codes in the LSI User Guides, but I found a helpful list readily compiled here:

MegaRAID Event Messages – Thomas-Krenn-Wiki

According to this list code 23 means that the scheduled Patrol read has completed, so this is a pretty harmless and expected message.

However, if disk or controller errors etc. are detected then other events will be generated.

I'm forwarding my host logs to vRealize Log Insight (you could also use another free syslog server) and have set up a custom alert there that sends me an e-mail when this message appears with an unexpected event code.

This works well for my "LSI MegaRAID SAS 9260-4i" adapter and should also work for other controllers using the megaraid_sas driver, but your mileage may vary.

I hope this helps

Andreas

Twitter: @VFrontDe, @ESXiPatches | https://esxi-patches.v-front.de | https://vibsdepot.v-front.de
0 Kudos
AlexJoda
Contributor
Contributor

Do you really want to search the vmkernel logs the whole day to find some LSI Controller errors? The must be a better solution for that! Are there any news regarding the updated CIM providers or any other LSI tools to fix this? We are still stuck with 6.5U1 and are not able to upgrade because of this....

0 Kudos
MysteryX
Contributor
Contributor

I managed to install MegaCLI and made some changes to an megaclisas-status file i found on the internet.

Now i'm getting this output:

-- Controller information --

-- ID | H/W Model                | RAM    | Temp | BBU    | Firmware    

c0    | LSI MegaRAID SAS 9260-8i | 512MB  | N/A  | Good   | FW: 12.15.0-0239

-- Array information --

-- ID | Type   |    Size |  Strpsz | Flags | DskCache |   Status |  OS Path | CacheCade |InProgress  

c0u0  | RAID-1 |   3637G |  1.0 MB | RA,WB |  Enabled |  Optimal |      N/A | None      |None        

c0u1  | RAID-1 |    232G |  256 KB | RA,WT |  Enabled |  Optimal |      N/A | None      |None        

-- Disk information --

-- ID  | Type | Drive Model                                        | Size     | Status          | Speed    | Temp | Slot ID  | LSI ID 

c0u0p0 | HDD  | WD-WCC7K4DJYN7RWDC WD40EFRX-68N32N0 82.00A82       | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 27C  | [252:1]  | 2      

c0u0p1 | HDD  | WD-WCC7K2XS6SYZWDC WD40EFRX-68N32N0 82.00A82       | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 27C  | [252:0]  | 3      

c0u1p0 | SSD  | S21PNSAG950842K Samsung SSD 850 EVO 250GB EMT01B6Q | 232.3 Gb | Online, Spun Up | 6.0Gb/s  | 28C  | [252:4]  | 4      

c0u1p1 | SSD  | S21PNSAG947444X Samsung SSD 850 EVO 250GB EMT01B6Q | 232.3 Gb | Online, Spun Up | 6.0Gb/s  | 28C  | [252:5]  | 5 

Now every reboot my files/modifications are gone again.

Can anyone put me in the right direction on how to create an vib file so they remain even after reboot.

Perhaps an nice web interfance would be nice...

Kind Regards,

0 Kudos
peetz
Leadership
Leadership

Broadcom/LSI provides a VIB file for the storcli utility (which is the successor to megacli).

You can download it here: https://docs.broadcom.com/docs/1.23.02_StorCLI

Twitter: @VFrontDe, @ESXiPatches | https://esxi-patches.v-front.de | https://vibsdepot.v-front.de
0 Kudos
JeffWDH
Contributor
Contributor

I was unable to get storcli or MegaCLI to display my H200 (crossflashed to 9211-8i in IR mode) adapter information:

    [root@hostname:~] esxcli storage core adapter list | grep mpt

    vmhba2    mpt2sas    link-n/a    sas.xx     (0000:02:00.0) LSI Logic / Symbios Logic Dell 6Gbps SAS HBA Adapter

   

   

    [root@hostname:~]  /opt/lsi/storcli/storcli /c0 show

    CLI Version = 007.0606.0000.0000 Mar 20, 2018

    Operating system = VMkernel 6.7.0

    Controller = 0

    Status = Failure

    Description = Controller 0 not found

   

    [root@hostname:/opt/lsi/MegaCLI] ./MegaCli -adpCount

   

    Controller Count: 0.

I found sas2ircu P20 on the Broadcom website which seems to work (as of 6.7.0 Update 1 Build 11675023):

Download location: https://docs.broadcom.com/docs-and-downloads/host-bus-adapters/host-bus-adapters-common-files/sas_sa...

Extract the vmware-esx-sas2ircu.vib from this file and upload it to your ESXi host. Install it using the following command (change path to match your datastore):

    [root@hostname:~] esxcli software vib install -v /vmfs/volumes/Datastore/vmware-esx-sas2ircu.vib --no-sig-check

   

    Installation Result

       Message: Operation finished successfully.

       Reboot Required: false

       VIBs Installed: LSI_bootbank_vmware-esx-sas2ircu_20.00.00.00-03

       VIBs Removed:

       VIBs Skipped:

I was then able to use sas2ircu to display my adapter information:

    [root@hostname:~] /opt/lsi/bin/sas2ircu 0 display

    LSI Corporation SAS2 IR Configuration Utility.

    Version 20.00.00.00 (2014.09.18)

    Copyright (c) 2008-2014 LSI Corporation. All rights reserved.

   

    Read configuration has been initiated for controller 0

    ------------------------------------------------------------------------

    Controller information

    ------------------------------------------------------------------------

      Controller type                         : SAS2008

      BIOS version                            : 7.39.02.00

      Firmware version                        : 20.00.07.00

      Channel description                     : 1 Serial Attached SCSI

      Initiator ID                            : 0

      Maximum physical devices                : 255

      Concurrent commands supported           : 1720

      Slot                                    : 4

      Segment                                 : 0

      Bus                                     : 2

      Device                                  : 0

      Function                                : 0

      RAID Support                            : Yes

    ------------------------------------------------------------------------

    IR Volume information

    ------------------------------------------------------------------------

    IR volume 1

      Volume ID                               : 286

      Status of volume                        : Okay (OKY)

      Volume wwid                             :

      RAID level                              : RAID1

      Size (in MB)                            : 113487

      Physical hard disks                     :

      PHY[0] Enclosure#/Slot#                 : 1:4

      PHY[1] Enclosure#/Slot#                 : 1:5

    ------------------------------------------------------------------------

    Physical device information

    ------------------------------------------------------------------------

    Initiator at ID #0

   

    Device is a Hard disk

      Enclosure #                             : 1

      Slot #                                  : 4

      SAS Address                             :

      State                                   : Optimal (OPT)

      Size (in MB)/(in sectors)               : 114473/234441647

      Manufacturer                            : ATA

      Model Number                            : KINGSTON SA400S3

      Firmware Revision                       : B1D1

      Serial No                               :

      GUID                                    :

      Protocol                                : SATA

      Drive Type                              : SATA_SSD

   

    Device is a Hard disk

      Enclosure #                             : 1

      Slot #                                  : 5

      SAS Address                             :

      State                                   : Optimal (OPT)

      Size (in MB)/(in sectors)               : 114473/234441647

      Manufacturer                            : ATA

      Model Number                            : KINGSTON SA400S3

      Firmware Revision                       : B1D1

      Serial No                               :

      GUID                                    :

      Protocol                                : SATA

      Drive Type                              : SATA_SSD

    ------------------------------------------------------------------------

    Enclosure information

    ------------------------------------------------------------------------

      Enclosure#                              : 1

      Logical ID                              :

      Numslots                                : 8

      StartSlot                               : 0

    ------------------------------------------------------------------------

    SAS2IRCU: Command DISPLAY Completed Successfully.

    SAS2IRCU: Utility Completed Successfully.

This doesn't help with the UI view but I'll grab this information via SSH and monitor my array this way.

0 Kudos
jefferson342
Contributor
Contributor

it probably on LSI update the driver when an existing driver stops working in the same major exsi release number, it's VMware fault, the existing working 6.5 driver shouldn't be prevented from working.

0 Kudos
ipv6
VMware Employee
VMware Employee

> as well as a ton of other non-storage hardware components

This should get cleaned up  in 6.5U3, PR PR 2297764 where LSI CIM Provider

classes are directly queried instead of base classes which dragged in non-storage

health related cim classes. (from ESX shell) visible as:

enum_instances CIM_Controller
             enum_instances CIM_Battery
             enum_instances CIM_StorageExtent
             enum_instances CIM_StorageVolume
             enum_instances CIM_SASSATAPort

vs:

LSIESG_PCIDevice

LSIESG_PhysicalCard

LSIESG_FirmwarePackageIdentity

LSIESG_MegaRAIDHBA

LSIESG_DiskDrive

LSIESG_DiskDriveProduct

Mike

ipv6
VMware Employee
VMware Employee

Then this KB would be of keen interest:  kb.vmware.com/s/article/52583

0 Kudos
ipv6
VMware Employee
VMware Employee

kb.vmware.com/s/article/52583  lists ESXi 6.7 depreciated/unsupported devices

Basically no support for HBA megaraid_sas, so would not expect CIM provider either.

Here's the search I used to locate Avago HBA cards that would be supported.

www.vmware.com/resources/compatibility/search.php?deviceCategory=io&details=1&partner=625&releases=3...

jamesmcewan
Contributor
Contributor

I'm experiencing the same issue as the OP and lots of other people who are running ESXi 6.5U2/6.7 with LSI Raid Controllers in their environments.

I am unable to view the status of the raid controller and individual drives in the Web Client or through the vSphere API; instead I see lots of unrelated hardware with 'Unknown' status.

This is preventing us from upgrading our ESXi hosts as we rely on this status for our monitoring. We query the vSphere API for the hardware status information and an alarm is triggered if the drives have failed or the raid is degraded.

After creating a case with VMware support and reading some of the other community posts regarding the issue (e.g. LSI SMIS на ESXi 6.7 ), I was led to believe this issue was with the LSI SMI-S Provider not supporting the latest version of ESXi and that this would be resolved in the next release (MegaRaid 7.9).

However, even after installing the latest version released on 25/3/2019 (VMW-ESX-6.5.0-lsiprovider-500.04.V0.73), I was still unable to see the correct hardware status information in vSphere. Broadcom support confirmed that they do not support the vCenter Web Client (even though it has worked in the past) and that I should contact VMware Support.

@ipv6 you seem to be aware of this issue and claim that this will be resolved in the next release of vSphere. Can you confirm that this is a vSphere related issue and that it will also be resolved in the next release of vSphere 6.7? Can you elaborate on how this issue arose in the first place and why it has taken so long to fix?

This should have been tested and discovered imo by VMware prior to the release of 6.5U2. Something as crucial as displaying drive status in the hardware health should not break for devices which are supported according to the HCL.

0 Kudos
jeffshead1
Contributor
Contributor

Same issue with LSI 9260-8i RAID Controller.

Wish Broadcom and/or VMWare would just give a straight answer so we know to wait or move to new hardware.

0 Kudos
clifford64
Contributor
Contributor

I contacted Broadcom support and the reply I got back is posted below. I tried the SMIS they provided, but it didn't seem to work for me. Let me know if you have any luck because I may not know what I am doing.

Yes we will come out with an SMIs provider or find the root cause.
6.5 was working fine and theoretically, the 6.5 SMIs provider should have worked.

But it is broken and no one seems to have a reliable fix at tthe moment.

But we are working on it.
We have had some success with this version:

ftp://tsupport:tsupport@ftp0.broadcom.com/private/LSA/vmware-smis.zip

You are more than welcome to try it.
If it does not work, just uninstall it and I will keep your case open until the final release comes out.

If to no avail, use storcli
Latest storcli for all OS:
https://docs.broadcom.com/docs/1.21.06_StorCLI

Thank you and let me know if you have any questions,
Storage Technical support
Broadcom Inc.
Joseph

0 Kudos
ilganna
Contributor
Contributor

Hi,

unfortunately the SMIS providers posted on the FTP address didn't worked for me Smiley Sad

FYI I have LSI MegaRAID SAS 9341-8i card and I tried with ESXi 6.7 U2.

Best regards

0 Kudos
plaurent
Contributor
Contributor

I just tried the SMIS providers posted on the FTP site as well, and they did not work for me either on 6.7 U2.

0 Kudos
martinlinder
Contributor
Contributor

I have installed the LSI Provider from Lenovo on 6.7U1

https://datacentersupport.lenovo.com/us/en/downloads/DS505420

with the LSI Storage Authority (replacement of Mega RAID Storage Manager) I can manage the LSI controller

Hardware is Supermicro Mainboard with LSI Controller

0 Kudos
ilganna
Contributor
Contributor

Tried with Lenovo SMIS providers (also drivers), but still not able to see drives/card status under Monitor -> Hardware -> Storage .

I'm also on Supermicro server HW (SYS-5028D-TN4T)

Thanks, best regards

0 Kudos
sleeplessnight
Contributor
Contributor

Have someone try to connect via Megaraid Storage Manager on a 6.7?

Install the MSM on a Virtual Windows Machine and try to connect over IP?


					
				
			
			
				
			
			
			
			
			
			
			
		
0 Kudos
jamesmcewan
Contributor
Contributor

I've tried to install the LSI Storage Authority (LSA) which, as I understand, is the replacement for Megaraid Storage Manager (MSM), and fully supported by LSI as the method for monitoring RAID controllers.

User Guide - LSI Storage Authority Software User Guide

My hope was that I would be able to monitor the raid status for my ESXi 6.5 and 6.7 hosts using this software.

However, I encountered problems when trying to add more than a single ESXi host to the LSI Storage Authority server. Apparently there is a bug which prevents you from being to monitor more than a single ESXi server per LSA instance:

https://www.broadcom.com/support/knowledgebase/1211221428056/how-to-monitor-esxi-6.7-with-lsa

The workaround is to install an LSA Server per ESXi host, which just doesn't scale if you have more than a handful of hosts. And to be honest, I'd rather SSH to each ESXi host and run storcli to monitor the raid status, that have to install a few hundred LSA servers...

At this point I'm starting to lose hope. It is so frustrating that neither VMware or LSI/Broadcom appear to be taking ownership of this issue. It's almost as though they don't understand the gravity of the problem.

0 Kudos
v0ha
Contributor
Contributor

https://www.vmware.com/resources/compatibility/pdf/vi_cim_guide.pdf here officially listed that 6.5u2 is the last version of LSI Provider support. Same boat here: controller is fully supported but vib not. So seems like moving back to 6.5

0 Kudos