Hi all,
My system:
Client version: 1.33.4
Client build number: 14110286
ESXi version: 6.0.0
ESXi build number: 15169789
Lenovo - RD550 (ThinkServer)
SN: S4F89540
Machine Type: 70CX
Model: 000EEA
RAID card: Lenovo ThinkServer RAID 720ix AnyRAID
BIOS 1.10.0
Deployment Manager 1.1.1
Linux Driver Bundle 1.1.1
System Manager 1.15.65376
Windows Driver Bundle 1.1.1
Problems:
1. Unable to retrieve information about health of physical disks in RAID arrays.
2. Receiving false positive information about degraded RAID arrays: there are no amber light on the panel, booting to RAID manager in BIOS reports all disks as healthy and all RAID arrays as optimal.
What I have tried:
https://kb.vmware.com/s/article/2040405
esxcli storage core device list
naa.600062b200b669701f5a9c8d135cc3d8
Display Name: Lenovo Serial Attached SCSI Disk (naa.600062b200b669701f5a9c8d135cc3d8)
Has Settable Display Name: true
Size: 7628800
Device Type: Direct-Access
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/disks/naa.600062b200b669701f5a9c8d135cc3d8
Vendor: Lenovo
Model: 720ix
Revision: 4.23
SCSI Level: 5
Is Pseudo: false
Status: degraded
Is RDM Capable: true
Is Local: false
Is Removable: false
Is SSD: false
Is VVOL PE: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: unknown
Attached Filters:
VAAI Status: unsupported
Other UIDs: vml.0200000000600062b200b669701f5a9c8d135cc3d8373230697820
Is Shared Clusterwide: true
Is Local SAS Device: false
Is SAS: true
Is USB: false
Is Boot USB Device: false
Is Boot Device: false
Device Max Queue Depth: 256
No of outstanding IOs with competing worlds: 32
Drive Type: logical
RAID Level: RAID5
Number of Physical Drives: 5
Protection Enabled: false
PI Activated: false
PI Type: 0
PI Protection Mask: NO PROTECTION
Supported Guard Types: NO GUARD SUPPORT
DIX Enabled: false
DIX Guard Type: NO GUARD SUPPORT
Emulated DIX/DIF Enabled: false
naa.500062b200b6697d
Display Name: Lenovo Serial Attached SCSI Enclosure Svc Dev (naa.500062b200b6697d)
Has Settable Display Name: true
Size: 0
Device Type: Enclosure Svc Dev
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/genscsi/naa.500062b200b6697d
Vendor: Lenovo
Model: 720ix
Revision: 0305
SCSI Level: 5
Is Pseudo: false
Status: on
Is RDM Capable: true
Is Local: false
Is Removable: false
Is SSD: false
Is VVOL PE: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: unknown
Attached Filters:
VAAI Status: unsupported
Other UIDs: vml.020d000000500062b200b6697d373230697820
Is Shared Clusterwide: true
Is Local SAS Device: false
Is SAS: true
Is USB: false
Is Boot USB Device: false
Is Boot Device: false
Device Max Queue Depth: 64
No of outstanding IOs with competing worlds: 32
Drive Type: physical
RAID Level: NA
Number of Physical Drives: 1
Protection Enabled: false
PI Activated: false
PI Type: 0
PI Protection Mask: NO PROTECTION
Supported Guard Types: NO GUARD SUPPORT
DIX Enabled: false
DIX Guard Type: NO GUARD SUPPORT
Emulated DIX/DIF Enabled: false
naa.600062b200b669701c243bb24a9f4d7b
Display Name: Lenovo Serial Attached SCSI Disk (naa.600062b200b669701c243bb24a9f4d7b)
Has Settable Display Name: true
Size: 2573568
Device Type: Direct-Access
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/disks/naa.600062b200b669701c243bb24a9f4d7b
Vendor: Lenovo
Model: 720ix
Revision: 4.23
SCSI Level: 5
Is Pseudo: false
Status: degraded
Is RDM Capable: true
Is Local: false
Is Removable: false
Is SSD: false
Is VVOL PE: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: unknown
Attached Filters:
VAAI Status: unsupported
Other UIDs: vml.0200000000600062b200b669701c243bb24a9f4d7b373230697820
Is Shared Clusterwide: true
Is Local SAS Device: false
Is SAS: true
Is USB: false
Is Boot USB Device: false
Is Boot Device: true
Device Max Queue Depth: 192
No of outstanding IOs with competing worlds: 32
Drive Type: logical
RAID Level: RAID5
Number of Physical Drives: 4
Protection Enabled: false
PI Activated: false
PI Type: 0
PI Protection Mask: NO PROTECTION
Supported Guard Types: NO GUARD SUPPORT
DIX Enabled: false
DIX Guard Type: NO GUARD SUPPORT
Emulated DIX/DIF Enabled: false
esxcfg-scsidevs -l
naa.500062b200b6697d
Device Type: Enclosure Svc Dev
Size: 0 MB
Display Name: Lenovo Serial Attached SCSI Enclosure Svc Dev (naa.500062b200b6697d)
Multipath Plugin: NMP
Console Device: /vmfs/devices/genscsi/naa.500062b200b6697d
Devfs Path: /vmfs/devices/genscsi/naa.500062b200b6697d
Vendor: Lenovo Model: 720ix Revis: 0305
SCSI Level: 5 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false Is SSD: false
Other Names:
vml.020d000000500062b200b6697d373230697820
VAAI Status: unsupported
naa.600062b200b669701c243bb24a9f4d7b
Device Type: Direct-Access
Size: 2573568 MB
Display Name: Lenovo Serial Attached SCSI Disk (naa.600062b200b669701c243bb24a9f4d7b)
Multipath Plugin: NMP
Console Device: /vmfs/devices/disks/naa.600062b200b669701c243bb24a9f4d7b
Devfs Path: /vmfs/devices/disks/naa.600062b200b669701c243bb24a9f4d7b
Vendor: Lenovo Model: 720ix Revis: 4.23
SCSI Level: 5 Is Pseudo: false Status: degraded
Is RDM Capable: true Is Removable: false
Is Local: false Is SSD: false
Other Names:
vml.0200000000600062b200b669701c243bb24a9f4d7b373230697820
VAAI Status: unsupported
naa.600062b200b669701f5a9c8d135cc3d8
Device Type: Direct-Access
Size: 7628800 MB
Display Name: Lenovo Serial Attached SCSI Disk (naa.600062b200b669701f5a9c8d135cc3d8)
Multipath Plugin: NMP
Console Device: /vmfs/devices/disks/naa.600062b200b669701f5a9c8d135cc3d8
Devfs Path: /vmfs/devices/disks/naa.600062b200b669701f5a9c8d135cc3d8
Vendor: Lenovo Model: 720ix Revis: 4.23
SCSI Level: 5 Is Pseudo: false Status: degraded
Is RDM Capable: true Is Removable: false
Is Local: false Is SSD: false
Other Names:
vml.0200000000600062b200b669701f5a9c8d135cc3d8373230697820
VAAI Status: unsupported
-----------------------------------------------
I can see 2 virtual disks and a physical disk (hot spare). I can see RAID level and number of physical drives but no information about individual disks.
No S.M.A.R.T. info logged into syslog either:
cat /var/log/syslog.log | grep smartd | wc -l
0
Not only I'm unable to check individual disks but I continue seeing false positives (Status: degraded) as well as this:
There were no amber lights on any of the caddies. I shut the server down, reseated all disks, booted into BIOS and confirmed in RAID settings all disks were reporting healthy and both arrays optimal.
Please advise.
Regards,
Adam
The following didn't help:
esxcli software vib install -d VMW-ESX-6.0.0-emulex-cim-provider-11.4.142.11-01-offline_bundle-6422346.zip
Eventually solved by:
esxcli software vib install -v lsi-mr3-7.702.51.00-1OEM.600.0.0.2768847.x86_64.vib
esxcli software vib install --no-sig-check -v vmware-esx-provider-lsiprovider.vib
Followed by a reboot.
I have also noticed the following error under Host -> Monitor -> Hardware -> Storage:
"The Small Footprint CIM Broker Daemon (SFCBD) is running, but no data has been reported. You may need to install a CIM provider for your storage adapter."
My list of modules:
esxcli system module list
ahci
cbt
cdp
chardevs
cmmds
cmmds_net
cmmds_resolver
cnic_register
deltadisk
dm
dma_mapper_iommu
dvfg-igmp
dvfilter
dvfilter-generic-fastpath
dvsdev
ehci-hcd
esxfw
etherswitch
filtmod
ftcpt
gss
hbr_filter
healthchk
heartbeat
hid
igb
iodm
ipfix
ipmi_devintf
ipmi_msghandler
ipmi_si_drv
iscsi_trans
iscsi_trans_compat_shim
iscsi_trans_incompat_shim
iscsi_vmk
lacp
libata_92
libata_9_2_0_0
libata_9_2_1_0
libata_9_2_2_0
libfc_92
libfc_9_2_0_0
libfc_9_2_1_0
libfcoe_92
libfcoe_9_2_0_0
libfcoe_9_2_1_0
lldp
lsi_mr3
lsom
lsomcommon
lvmdriver
migrate
netioc
netsched
nfs41client
nfsclient
nmp
plog
procfs
procMisc
random
rdt
shaper
sunrpc
svmmirror
swapobj
tcpip4
teamcheck
tracing
ufs
usb
usb-storage
user
vdfm
vfat
vfc
vflash
virsto
vlanmtucheck
vmci
vmfs3
vmkapei
vmkapi_mgmt
vmkapi_v2_0_0_0_dvfilter_shim
vmkapi_v2_0_0_0_iscsi_shim
vmkapi_v2_0_0_0_nmp_shim
vmkapi_v2_0_0_0_vmkernel_shim
vmkapi_v2_1_0_0_dvfilter_shim
vmkapi_v2_1_0_0_iscsi_shim
vmkapi_v2_1_0_0_nmp_shim
vmkapi_v2_1_0_0_vmkernel_shim
vmkapi_v2_2_0_0_dvfilter_shim
vmkapi_v2_2_0_0_iscsi_shim
vmkapi_v2_2_0_0_mgmt_shim
vmkapi_v2_2_0_0_nmp_shim
vmkapi_v2_2_0_0_vmkernel_shim
vmkernel
vmkibft
vmklink_mpi
vmklinux_9
vmklinux_9_2_0_0
vmklinux_9_2_1_0
vmklinux_9_2_2_0
vmklinux_9_2_3_0
vmkplexer
vmkstatelogger
vmw_psp_fixed
vmw_psp_lib
vmw_psp_mru
vmw_psp_rr
vmw_satp_default_aa
vmw_satp_local
vprobe
vsan
vsanapi
vsanbase
vsanutil
xhci
"Is Loaded" and "Is Enabled" is true for all.
Is there something missing or in need for an update that's causing my issue, e.g:
https://support.lenovo.com/gb/en/downloads/ds120657
or
https://support.lenovo.com/us/en/downloads/ds041848
?
My server's serial number: S4F89540
Followed a suggestion from Lenovo support I installed CIM provider which was reporting as missing:
[root@esxi60:/var/log/vmware] esxcli software vib install -d VMW-ESX-6.0.0-emulex-cim-provider-11.4.142.11-01-offline_bundle-6422346.zip
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: EMU_bootbank_emulex-cim-provider_11.4.142.11-01OEM.600.0.0.2768847
VIBs Removed:
VIBs Skipped:
esxcli software vib list | grep cim
emulex-cim-provider 11.4.142.11-01OEM.600.0.0.2768847 EMU VMwareAccepted 2020-11-04
This was followed by a reboot but unfortunately my problem persists. The CIM provider still reports to be missing. ESXi can only see virtual drives in degraded state (which is a false positive) and hasn't got a clue about individual disks and their health
The following didn't help:
esxcli software vib install -d VMW-ESX-6.0.0-emulex-cim-provider-11.4.142.11-01-offline_bundle-6422346.zip
Eventually solved by:
esxcli software vib install -v lsi-mr3-7.702.51.00-1OEM.600.0.0.2768847.x86_64.vib
esxcli software vib install --no-sig-check -v vmware-esx-provider-lsiprovider.vib
Followed by a reboot.