We have multiple ESXi 5.1 hosts attached to an IBM XIV. We are exposing LUNs for use as RDM's. In general this works great. However, we are in the midst of migrating some clusters from iSCSI to FC and had an issue where the still-active iSCSI attached node had some delays and the cluster failed over to the other cluster node which was attached via FC to the same LUNs.
This is all MSCS.
On the ESX host where the active node is now, we're getting lots of these in the logs:
2013-06-06T23:08:44.812Z cpu22:5582616)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x2a (0x41244732b300, 5582610) to dev "eui.001738000f862592" on path "vmhba2:C0:T0:L40" Failed: H:0x1 D:0x0 P:0x0 Possible sense data: 0x6 0x2a 0x5. Act:FAILOVER
2013-06-06T23:08:44.812Z cpu22:5582616)WARNING: NMP: nmp_DeviceRetryCommand:133:Device "eui.001738000f862592": awaiting fast path state update for failover with I/O blocked. No prior reservation exists on the device.
2013-06-06T23:08:44.812Z cpu22:5582616)WARNING: NMP: nmp_DeviceStartLoop:721:NMP Device "eui.001738000f862592" is blocked. Not starting I/O from device.
2013-06-06T23:08:45.032Z cpu22:6859493)WARNING: NMP: nmpDeviceAttemptFailover:599:Retry world failover device "eui.001738000f862592" - issuing command 0x41244732b300
2013-06-06T23:08:45.034Z cpu16:5574422)NMP: nmpCompleteRetryForPath:321: Retry world recovered device "eui.001738000f862592"
The LUNs associated with these errors are experiencing high latency.
Looking at the sense data codes, this seems to be a UNIT ATTENTION REGISTRATION PREEMPTED message. I don't know what that means though, but seems to have something to do with SCSI-3 reservations. There are lots of these messages scrolling by very fast -- and they hit every one of the four paths we have to this LUN in succession.
Other clusters and LUNs are still working ok.
Any thoughts?
