ESXi, sluggish performance

[i976]

Hi everyone

I'm having an ESXi Host here with sluggish performance. I have collected the mertics of this host:

The host is a HPE ML350 G9 with SAS 15k Disks.

vmware -v
VMware ESXi 6.7.0 build-8169922

esxcli storage core adapter list
vmhba0 vmw_ahci link-n/a sata.vmhba0 (0000:00:1f.2) Intel Corporation Wellsburg AHCI Controller
vmhba1 nhpsa link-n/a sas.5001438040982450 (0000:03:00.0) Hewlett-Packard Company Smart Array P440ar

vmkload_mod -s nhpsa | grep Version
Version: 2.0.24-1OEM.650.0.0.4598673

esxcli network nic list
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
------ ------------ ------ ------------ ----------- ----- ------ ----------------- ---- -------------------------------------------------------
vmnic0 0000:02:00.0 ntg3 Up Up 1000 Full 70:10:6f:xx:xx:xx 1500 Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic1 0000:02:00.1 ntg3 Up Up 1000 Full 70:10:6f:xx:xx:xx 1500 Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic2 0000:02:00.2 ntg3 Up Up 1000 Full 70:10:6f:xx:xx:xx 1500 Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic3 0000:02:00.3 ntg3 Up Up 1000 Full 70:10:6f:xx:xx:xx 1500 Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet

usr/lib/vmware/vm-support/bin/nicinfo.sh | less
rxMulticastPkts : 16156839
rxBroadcastPkts : 41616631
rxCRCErrors : 0
rxFrameAlignErrors : 0
rxXon : 0
rxXoff : 0
rxMACCtlFrms : 0
rxXoffEntry : 0
rxLengthErrors : 0
rxJabber : 0
rxUndersize : 0
rxOverflowErrors : 259
rxDrops : 0
rxErrors : 0

Driver: ntg3
Firmware Version: bc 1.46 ncsi 1.4.18.0
Version: 4.1.3.0
Link Detected: true
Link Status: Up
Name: vmnic0
PHY Address: 0
Pause Autonegotiate: true
Pause RX: true
Pause TX: true
Supported Ports: TP
Supports Auto Negotiation: true
Supports Pause: true
Supports Wakeon: true
Transceiver: internal
Virtual Address: 00:50:56:58:84:05
Wakeon: MagicPacket(tm)

See also the attached pictures. I cannot seem to pinpoint this issue...is nhpsa dodgy?

[a_p_]

Just a quick thought.
Did you already check the Smart Array's controller cache battery. A bad battery will cause the controller to switch to write-trough mode, which is way slower that the usual write-back mode.

André

[i976]

Thank you for your answer.

/opt/smartstorageadmin/ssacli/bin/ssacli ctrl all show detail

Smart Array P440ar in Slot 0 (Embedded)

Note: Predictive Spare Activation Mode is enabled, physical drives that are in predictive failure state will not be available for use as data or spare drives.

Bus Interface: PCI
Slot: 0
Serial Number: X
Cache Serial Number: X
RAID 6 (ADG) Status: Enabled
Controller Status: OK
Hardware Revision: B
Firmware Version: 6.30-0
Rebuild Priority: High
Expand Priority: Medium
Surface Scan Delay: 3 secs
Surface Scan Mode: Idle
Parallel Surface Scan Supported: Yes
Current Parallel Surface Scan Count: 1
Max Parallel Surface Scan Count: 16
Queue Depth: Automatic
Monitor and Performance Delay: 60 min
Elevator Sort: Enabled
Degraded Performance Optimization: Disabled
Inconsistency Repair Policy: Disabled
Wait for Cache Room: Disabled
Surface Analysis Inconsistency Notification: Disabled
Post Prompt Timeout: 15 secs
Cache Board Present: True
Cache Status: OK
Cache Ratio: 20% Read / 80% Write
Drive Write Cache: Disabled
Total Cache Size: 2.0
Total Cache Memory Available: 1.8
No-Battery Write Cache: Disabled
SSD Caching RAID5 WriteBack Enabled: True
SSD Caching Version: 2
Cache Backup Power Source: Batteries
Battery/Capacitor Count: 1
Battery/Capacitor Status: OK
SATA NCQ Supported: True
Spare Activation Mode: Activate on physical drive predictive failure
Controller Temperature (C): 36
Cache Module Temperature (C): 26
Number of Ports: 2 Internal only
Encryption: Not Set
Express Local Encryption: False
Driver Name: nhpsa
Driver Version: 2.0.24-1OEM
PCI Address (Domain🚌Device.Function): 0000:03:00.0
Negotiated PCIe Data Rate: PCIe 3.0 x8 (7880 MB/s)
Controller Mode: RAID
Pending Controller Mode: RAID
Port Max Phy Rate Limiting Supported: False
Latency Scheduler Setting: Disabled
Current Power Mode: MaxPerformance
Survival Mode: Enabled
Host Serial Number: 
Sanitize Erase Supported: True
Primary Boot Volume: None
Secondary Boot Volume: None

Seems to be ok? What do you think?

[i976]

The logical drives look OK too, I think...

Smart Array P440ar in Slot 0 (Embedded)

Array B

Logical Drive: 2
Size: 372.58 GB
Fault Tolerance: 1
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 256 KB
Status: OK
Unrecoverable Media Errors: None
MultiDomain Status: OK
Caching: Disabled
Unique Identifier: 600508B1001CEBF60A9D85DE3B7C3BA9
Logical Drive Label: 06E26405PDNLH0BRH558XD4ACB
Mirror Group 1:
physicaldrive 2I:6:7 (port 2I:box 6:bay 7, SAS SSD, 400 GB, OK)
Mirror Group 2:
physicaldrive 2I:6:8 (port 2I:box 6:bay 8, SAS SSD, 400 GB, OK)
Drive Type: Data
LD Acceleration Method: Smart Path

/opt/smartstorageadmin/ssacli/bin/ssacli ctrl slot=0 ld 1 show

Smart Array P440ar in Slot 0 (Embedded)

Array A

Logical Drive: 1
Size: 2.18 TB
Fault Tolerance: 5
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 1024 KB
Status: OK
Unrecoverable Media Errors: None
MultiDomain Status: OK
Caching: Enabled
Parity Initialization Status: Initialization Completed
Unique Identifier: X
Logical Drive Label: X
Drive Type: Data
LD Acceleration Method: Controller Cache

[angelbach69]

Same experience... same configuration! Solved anyway? thanks