Hi,
I have two dual arrays P2000fc G3 with extra enclosures. They are connected with six ESX hosts. I've recently updated all my infrastructure to vSphere 5 and ESXi 5.0 (rev. 504890).
Each array is configured as active/active and I use only one vdisk per controller (first is raid 10 (16 SAS hdd), second is raid 50 (32 SAS)). Each vdisk has only one volume. On two raid 10 LUNs we have about 7-8 machines per LUN (DC, web server, dhcp, monitoring servers, etc.) on raid 50 Luns we store 5 machines per LUN (MS SQL, vSphere, Linux repositiory, etc.).
I've recently upgraded all firmware:
On each esx hosts I use :
Path selection is changed to Round Robin (VMW_SATP_ALUA) and hardware acceleration is supported. Now all datastores are recreated (VMFS 5.0). When I checked vmkerner.log and I saw many warning:
2011-12-07T10:39:05.590Z cpu4:4100)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x41240115abc0) to dev "naa.600c0ff000115e59bbc9cc4e01000000" on path "vmhba0:C0:T2:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.Act:NONE
2011-12-07T10:39:05.590Z cpu4:4100)ScsiDeviceIO: 2305: Cmd(0x41240115abc0) 0x2a, CmdSN 0x42822 to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:10.651Z cpu20:4116)ScsiDeviceIO: 2305: Cmd(0x412441842e40) 0x2a, CmdSN 0x428f2 to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:15.713Z cpu6:4102)ScsiDeviceIO: 2305: Cmd(0x4124013342c0) 0x2a, CmdSN 0x428fc to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:20.876Z cpu10:4106)ScsiDeviceIO: 2305: Cmd(0x412401606e40) 0x2a, CmdSN 0x42abb to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:21.474Z cpu10:4106)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x4124015eb080) to dev "naa.600c0ff000d876f83c62104d01000000" on path "vmhba0:C0:T1:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.Act:EVAL
2011-12-07T10:39:21.474Z cpu10:4106)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600c0ff000d876f83c62104d01000000" state in doubt; requested fast path state update...
2011-12-07T10:39:21.474Z cpu10:4106)ScsiDeviceIO: 2305: Cmd(0x4124015eb080) 0x28, CmdSN 0x2bfbf to dev "naa.600c0ff000d876f83c62104d01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2011-12-07T10:39:26.343Z cpu1:4097)ScsiDeviceIO: 2305: Cmd(0x412401383080) 0x2a, CmdSN 0x42e0f to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:31.708Z cpu8:4104)ScsiDeviceIO: 2305: Cmd(0x4124015ce280) 0x2a, CmdSN 0x430d6 to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:36.769Z cpu10:4106)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x4124003bae80) to dev "naa.600c0ff000115e59bbc9cc4e01000000" on path "vmhba0:C0:T2:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.Act:NONE
2011-12-07T10:39:36.769Z cpu10:4106)ScsiDeviceIO: 2305: Cmd(0x4124003bae80) 0x2a, CmdSN 0x430ee to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:41.932Z cpu8:4104)ScsiDeviceIO: 2305: Cmd(0x41240153d180) 0x2a, CmdSN 0x4312a to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:43.765Z cpu16:4112)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x8a (0x412440368900) to dev "naa.600c0ff000115553687bcb4e01000000" on path "vmhba0:C0:T5:L1" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.Act:NONE
2011-12-07T10:39:43.765Z cpu16:4112)ScsiDeviceIO: 2305: Cmd(0x412440368900) 0x8a, CmdSN 0x800e004d to dev "naa.600c0ff000115553687bcb4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:46.007Z cpu5:4101)ScsiDeviceIO: 2305: Cmd(0x4124003b9080) 0x2a, CmdSN 0x800e0025 to dev "naa.600c0ff000d876f83c62104d01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:46.007Z cpu8:4104)ScsiDeviceIO: 2305: Cmd(0x412401600b40) 0x2a, CmdSN 0x800e001b to dev "naa.600c0ff000d876f83c62104d01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:46.994Z cpu4:4100)ScsiDeviceIO: 2305: Cmd(0x41240112af80) 0x2a, CmdSN 0x43213 to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
2011-12-07T10:39:48.705Z cpu0:4120)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600c0ff000115e59bbc9cc4e01000000" state in doubt; requested fast path state update...
2011-12-07T10:39:48.705Z cpu0:4120)ScsiDeviceIO: 2305: Cmd(0x4124014d2c00) 0x28, CmdSN 0x43695 to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2011-12-07T10:39:57.624Z cpu17:4113)ScsiDeviceIO: 2305: Cmd(0x41244036bc00) 0x2a, CmdSN 0x437d7 to dev "naa.600c0ff000115e59bbc9cc4e01000000" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x47 0x0.
I've observed that some machines have poor performance. I think this problem is correlated with warnings which I attached. Should I worry about this logs?
Hi there,
same probleme here. But its the P2000 G3 iSCSI 1 GBe:
2012-02-07T16:04:05.171Z cpu5:5777)ScsiDeviceIO: 2305: Cmd(0x412400773b80) 0x28, CmdSN 0x8000003f to dev "naa.600c0ff0001302cd1ff1114f01000000" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x20 0x0.
2012-02-07T16:04:05.175Z cpu5:5777)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x41240071d800) to dev "naa.600c0ff0001302cd1ff1114f01000000" on path "vmhba37:C4:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.Act:EVAL
2012-02-07T16:04:05.178Z cpu5:5777)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x412400778540) to dev "naa.600c0ff0001302cd1ff1114f01000000" on path "vmhba37:C4:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.Act:EVAL
2012-02-07T16:04:05.182Z cpu4:2052)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x41240079bcc0) to dev "naa.600c0ff0001302cd1ff1114f01000000" on path "vmhba37:C4:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x20 0x0.Act:EVAL
2012-02-07T16:04:05.182Z cpu4:2052)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x412400769d00) to dev "naa.600c0ff0001302cd1ff1114f01000000" on path "vmhba37:C4:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x20 0x0.Act:EVAL
2012-02-07T16:04:05.182Z cpu4:2052)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x412400703740) to dev "naa.600c0ff0001302cd1ff1114f01000000" on path "vmhba37:C4:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x20 0x0.Act:EVAL
2012-02-07T16:04:05.184Z cpu5:5777)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x28 (0x412400791a40) to dev "naa.600c0ff0001302cd1ff1114f01000000" on path "vmhba37:C4:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x2 0x3a 0x1.Act:EVAL
Sometimes i've a latency approx. 50 ms to the target! Same firmware of the P2000 and i'm using Round-Robin too.
