R430 with Dell customized 6.7 U3 | Taking time while create local vmfs datastore

[ManivelR]

Hi All,

I have a server with below specs.Server is running with dell customized 6.7 U3.ESXi OS sits on vflash(4 GB SD card). BIOS,LC and HBA 330 mini fw updated.

R430

HBA 330 mini and the fw version is "16.17.00.05"

BIOS version is 2.11.0 and LC version is "2.40.40.40"

8 SSD disks(Seagate Barracud & Adata SU 800)

When im trying to create any local vmfs datastore with any SSD disks,it is taking more than 5 mins to complete it also loading time is very high whenever i navigate.

I did not get any clue where is the problem and what is causing this delay? Any ideas?

VMkernel logs:-

2020-10-14T07:09:47.006Z cpu32:2100134)Partition: 676: Read from primary gpt table failed on "naa.5000c500bb0f0e92".

2020-10-14T07:09:47.454Z cpu0:2097235)ScsiDeviceIO: 3469: Cmd(0x45ba816cc740) 0x85, CmdSN 0x12 from world 2100134 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.

2020-10-14T07:10:08.599Z cpu49:2100558)WARNING: UserMem: 5073: openssl: Unable to create heap mmInfo: Already exists start 0x7f0e199000 end 0x7f0e1ba000

2020-10-14T07:10:10.007Z cpu10:2100561)WARNING: UserMem: 5073: openssl: Unable to create heap mmInfo: Already exists start 0x4ecbb41000 end 0x4ecbb62000

2020-10-14T07:10:23.697Z cpu20:2098137)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-10-14T07:10:27.657Z cpu2:2097237)ScsiDeviceIO: 3483: Cmd(0x45ba816cc740) 0x9e, CmdSN 0x1554 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:10:27.657Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a7a89a880):cmdSN(0x1554):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:11:07.658Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x45ba816cc740) 0x25, CmdSN 0x1555 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:11:07.658Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a7a89a900):cmdSN(0x1555):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:11:47.661Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x45ba816cc740) 0x1a, CmdSN 0x1556 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:11:47.661Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a7a89a840):cmdSN(0x1556):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:11:47.661Z cpu36:2100134)WARNING: ScsiDeviceIO: 10789: READ CAPACITY on device "naa.5000c500bb0f0de6" from Plugin "NMP" failed. I/O error

2020-10-14T07:11:47.679Z cpu36:2100134)Partition: 676: Read from primary gpt table failed on "naa.5000c500bb0f0fe5".

2020-10-14T07:12:03.747Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x459a81dcf680) 0x12, CmdSN 0x157c from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:12:03.748Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a5321bcc8):cmdSN(0x157c):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:12:03.748Z cpu22:2097764)ScsiVmas: 814: Inquiry for VPD page 00 to device naa.5000c500bb0f0de6 failed with error Timeout

2020-10-14T07:12:43.748Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x459a81dcf680) 0x12, CmdSN 0x1596 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:12:43.748Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a5321bcd8):cmdSN(0x1596):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:16:23.697Z cpu39:2098212)DVFilter: 6054: Checking disconnected filters for timeouts

[root@localhost:~]

Thanks,

Manivel RR

[nachogonzalez]

Hey, hope you are doing fine:

You VMkernel logs is plagued with SCSI sense codes indicating aborts

2020-10-14T07:10:27.657Z cpu2:2097237)ScsiDeviceIO: 3483: Cmd(0x45ba816cc740) 0x9e, CmdSN 0x1554 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:10:27.657Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a7a89a880):cmdSN(0x1554):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:11:07.658Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x45ba816cc740) 0x25, CmdSN 0x1555 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:11:07.658Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a7a89a900):cmdSN(0x1555):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:11:47.661Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x45ba816cc740) 0x1a, CmdSN 0x1556 from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2020-10-14T07:11:47.661Z cpu18:2097804)WARNING: lsi_msgpt3: _scsih_abort:770: lsi_msgpt3_0:C0:T8:L0:cmdInitiator(0x451a7a89a840):cmdSN(0x1556):worldId(0x0): scmd not found in scsi_lookup to abort

2020-10-14T07:11:47.661Z cpu36:2100134)WARNING: ScsiDeviceIO: 10789: READ CAPACITY on device "naa.5000c500bb0f0de6" from Plugin "NMP" failed. I/O error

2020-10-14T07:11:47.679Z cpu36:2100134)Partition: 676: Read from primary gpt table failed on "naa.5000c500bb0f0fe5".

2020-10-14T07:12:03.747Z cpu18:2097253)ScsiDeviceIO: 3483: Cmd(0x459a81dcf680) 0x12, CmdSN 0x157c from world 0 to dev "naa.5000c500bb0f0de6" failed H:0x5 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

Essentially what this says is that the host's driver is aborting SCSI commands between the driver and the storage array.
As I understood you are trying to configure all this disks locally. Please reply the following questions:

1. Are you sure your firmware/drivers are up to date?
VMware Compatibility Guide - I/O Device Search

2. Have you configured RAID in the server?
3. Are you sure the disks are properly connected and the controller is working properly?

Warm regards

[ManivelR]

Thanks for your response.

1. Are you sure your firmware/drivers are up to date? 
VMware Compatibility Guide - I/O Device Search

Im using dell customized 6.7 U3 image and also updated the version already.

HBA 330 mini and the fw version is "16.17.00.05"

BIOS version is 2.11.0 and LC version is "2.40.40.40"

2. Have you configured RAID in the server?

No Raid configured as this is testing ESXi server.

3. Are you sure the disks are properly connected and the controller is working properly?

As communicated earlier,I have 8 SSD disks in my server and all these 8 SSD disks are connected with DELL HBA 330 controller.

Whenever I create any local VMFS DS with any SSD disk(out of 8 disks),the result is same(I mean it is taking longer time.)

Thanks,

Manivel RR

[IRIX201110141]

For the HBA330 is the most rescent FW 16.17.01.00. They fixed T10 protection related stuff.

iDRAC/LCC is 2.75.75.75. But more important than iDRAC is the drive FW.

You should consider to upgrade all FWs and not only partially.

Do you have the problem when only ONE drive is connected to the HBA330?

Regards,
Joerg