2021-04-26T00:35:44.485Z cpu12:2097360)Queued:1 2021-04-26T00:35:44.485Z cpu2:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout 2021-04-26T00:35:44.486Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c631c0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:35:48.735Z cpu12:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0dd6ac0) 0x28, cmdId.initiator=0x4305118d2080 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:35:48.735Z cpu12:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:35:48.735Z cpu12:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0dd6ac0) 0x28, cmdId.initiator=0x4305118d2080 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:35:48.735Z cpu12:2097360)Queued:1 2021-04-26T00:35:48.736Z cpu9:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1. 2021-04-26T00:35:55.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c14fc0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:35:55.487Z cpu9:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:36:04.486Z cpu4:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times 2021-04-26T00:36:06.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c48dc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:36:06.486Z cpu3:2097361)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x0 (0x45d8c0c48dc0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:36:06.486Z cpu3:2097361)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x453891e1bc58 CmdSN 0x0 2021-04-26T00:36:06.486Z cpu11:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:36:24.485Z cpu12:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e2127f40) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8d9 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:36:24.485Z cpu12:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:36:24.485Z cpu12:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e2127f40) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8d9 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:36:24.485Z cpu12:2097360)Queued:2 2021-04-26T00:36:24.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c631c0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:36:28.736Z cpu12:2097198)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x9e, cmdId.initiator=0x45389639a3f8 CmdSN 0x1e8db from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:1 2021-04-26T00:36:28.736Z cpu12:2097198)Queued:0 2021-04-26T00:36:28.737Z cpu12:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0cc62c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8da from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:36:28.738Z cpu12:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:36:28.738Z cpu12:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0cc62c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8da from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:36:28.738Z cpu12:2097360)Queued:1 2021-04-26T00:36:28.738Z cpu10:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure 2021-04-26T00:36:34.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37e7480) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:36:34.486Z cpu10:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:36:45.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d112c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:36:45.485Z cpu9:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:37:04.485Z cpu4:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 5 times 2021-04-26T00:37:08.739Z cpu13:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x25, cmdId.initiator=0x45389639a4a8 CmdSN 0x1e8e0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:37:08.739Z cpu13:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x25 (0x45d8e375f580, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:37:08.739Z cpu13:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a4a8 CmdSN 0x1e8e0 2021-04-26T00:37:08.739Z cpu13:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:37:08.739Z cpu13:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x25, cmdId.initiator=0x45389639a4a8 CmdSN 0x1e8e0 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:37:08.739Z cpu13:2097360)Queued:0 2021-04-26T00:37:19.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3721780) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:37:19.485Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:37:48.740Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a3e8 CmdSN 0x1e8e3 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:37:48.740Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:37:48.740Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a3e8 CmdSN 0x1e8e3 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:37:48.740Z cpu10:2097360)Queued:0 2021-04-26T00:37:59.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c14fc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:37:59.486Z cpu8:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:38:04.487Z cpu3:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times 2021-04-26T00:38:28.741Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a4c8 CmdSN 0x1e8e4 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:38:28.741Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8e375f580, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:38:28.741Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a4c8 CmdSN 0x1e8e4 2021-04-26T00:38:28.741Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:38:28.741Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a4c8 CmdSN 0x1e8e4 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:38:28.741Z cpu10:2097360)Queued:0 2021-04-26T00:38:39.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e368cb80) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:38:39.486Z cpu10:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:39:04.485Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 1 times 2021-04-26T00:39:08.744Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x28, cmdId.initiator=0x430511976f00 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:39:08.744Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8e375f580, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:39:08.744Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511976f00 CmdSN 0x1 2021-04-26T00:39:08.744Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:39:08.744Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x28, cmdId.initiator=0x430511976f00 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:39:08.744Z cpu9:2097360)Queued:0 2021-04-26T00:39:08.744Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1. 2021-04-26T00:39:08.744Z cpu8:2100597)LVM: 6192: Could not open device mpx.vmhba32:C0:T0:L0:7, vol [60724578-09db3a26-f7cb-000af7abb22c, 60724578-09db3a26-f7cb-000af7abb22c, 1]: Timeout 2021-04-26T00:39:08.745Z cpu8:2100597)Vol3: 2129: Could not open device 'mpx.vmhba32:C0:T0:L0:7' for volume open: Not found 2021-04-26T00:39:08.745Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 1 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 0 gen 0 :Not found 2021-04-26T00:39:08.745Z cpu8:2100597)WARNING: Fil3: 1534: Failed to reserve volume f533 28 1 6072457a 332348ce a005c7a 2cb2abf7 0 0 0 0 0 0 0 2021-04-26T00:39:08.745Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 2 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 4 gen 1 :Not found 2021-04-26T00:39:19.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c5a5c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:39:19.486Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:39:48.744Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0ca50c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8e7 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:39:48.744Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:39:48.744Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0ca50c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8e7 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:39:48.744Z cpu9:2097360)Queued:1 2021-04-26T00:39:48.744Z cpu8:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure 2021-04-26T00:39:48.748Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x9e, cmdId.initiator=0x45389639a458 CmdSN 0x1e8e8 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:39:48.748Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x9e, cmdId.initiator=0x45389639a458 CmdSN 0x1e8e8 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:39:48.748Z cpu9:2097360)Queued:0 2021-04-26T00:39:59.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d126c0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:39:59.485Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:40:04.485Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 4 times 2021-04-26T00:40:14.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9d3c0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:40:14.486Z cpu6:2097361)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x0 (0x45d8c0c9d3c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:40:14.486Z cpu6:2097361)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x453885e1bc58 CmdSN 0x0 2021-04-26T00:40:14.486Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:40:28.750Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8e9 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:40:28.750Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:40:28.750Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8e9 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:40:28.750Z cpu10:2097360)Queued:1 2021-04-26T00:40:39.489Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c7e4c0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:40:39.489Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:40:44.488Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37d7f80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:40:44.489Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e8ee from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:40:44.489Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:40:44.489Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e8ee from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:40:44.489Z cpu8:2097360)Queued:1 2021-04-26T00:40:44.489Z cpu2:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout 2021-04-26T00:40:55.486Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3660580) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:40:55.486Z cpu9:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:41:04.485Z cpu6:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 5 times 2021-04-26T00:41:08.750Z cpu14:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8ef from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:41:08.750Z cpu14:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c9c4c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:41:08.750Z cpu14:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a448 CmdSN 0x1e8ef 2021-04-26T00:41:08.750Z cpu14:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:41:08.750Z cpu14:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8ef from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:41:08.750Z cpu14:2097360)Queued:1 2021-04-26T00:41:19.487Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37a4680) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:41:19.487Z cpu8:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:41:24.488Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37d7f80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:41:24.490Z cpu14:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8f0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:41:24.490Z cpu14:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:41:24.490Z cpu14:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8f0 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:41:24.490Z cpu14:2097360)Queued:1 2021-04-26T00:41:35.488Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37aaa80) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:41:35.488Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:41:48.750Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e8f1 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:41:48.750Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:41:48.751Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e8f1 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:41:48.751Z cpu8:2097360)Queued:0 2021-04-26T00:41:59.485Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c4bfc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:41:59.485Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:42:04.485Z cpu6:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times 2021-04-26T00:42:28.751Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:42:28.751Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8c0c9c4c0, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:42:28.751Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511963440 CmdSN 0x1 2021-04-26T00:42:28.751Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:42:28.751Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:42:28.751Z cpu9:2097360)Queued:0 2021-04-26T00:42:28.751Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1. 2021-04-26T00:42:39.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d329c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:42:39.487Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:43:04.487Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 1 times 2021-04-26T00:43:08.752Z cpu8:2097194)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x9e, cmdId.initiator=0x45389639a458 CmdSN 0x1e8f9 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:1 2021-04-26T00:43:08.752Z cpu8:2097194)Queued:0 2021-04-26T00:43:10.485Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8f8 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:43:10.486Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x35 (0x45d8c0c9c4c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:43:10.486Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45388621be18 CmdSN 0x1e8f8 2021-04-26T00:43:10.486Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:43:10.486Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8f8 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:43:10.486Z cpu9:2097360)Queued:1 2021-04-26T00:43:10.486Z cpu8:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure 2021-04-26T00:43:20.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e372a380) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:43:20.486Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:43:48.755Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c2c1c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8fa from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:43:48.755Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:43:48.755Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8fa from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:43:48.755Z cpu9:2097360)Queued:0 2021-04-26T00:43:59.487Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e368da80) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:43:59.487Z cpu10:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:44:04.487Z cpu5:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times 2021-04-26T00:44:28.757Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8fd from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:44:28.757Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c2c1c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:44:28.757Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a448 CmdSN 0x1e8fd 2021-04-26T00:44:28.757Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:44:28.757Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8fd from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:44:28.757Z cpu8:2097360)Queued:1 2021-04-26T00:44:28.759Z cpu7:2101729)ALERT: hostd performance has degraded due to high system latency 2021-04-26T00:44:39.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e377b280) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:44:39.486Z cpu8:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:45:01.981Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x9e, cmdId.initiator=0x45389269a7d8 CmdSN 0x1e909 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:45:01.981Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:45:01.981Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x9e, cmdId.initiator=0x45389269a7d8 CmdSN 0x1e909 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:45:01.981Z cpu10:2097360)Queued:1 2021-04-26T00:45:04.486Z cpu2:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 2 times 2021-04-26T00:45:08.757Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e90a from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:45:08.757Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c2c1c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:45:08.757Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a528 CmdSN 0x1e90a 2021-04-26T00:45:08.757Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:45:08.757Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e90a from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:45:08.757Z cpu10:2097360)Queued:2 2021-04-26T00:45:12.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37ece80) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:45:12.487Z cpu11:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:45:23.490Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3626380) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:45:23.490Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:45:41.981Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x25, cmdId.initiator=0x45389269a888 CmdSN 0x1e90d from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:45:41.981Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:45:41.981Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x25, cmdId.initiator=0x45389269a888 CmdSN 0x1e90d from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:45:41.981Z cpu11:2097360)Queued:2 2021-04-26T00:45:44.488Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e912 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:45:44.488Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:45:44.488Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e912 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:45:44.488Z cpu11:2097360)Queued:2 2021-04-26T00:45:44.488Z cpu2:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout 2021-04-26T00:45:44.491Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d00ec0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:45:48.759Z cpu9:2097195)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x28, cmdId.initiator=0x4305119491c0 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:1 2021-04-26T00:45:48.759Z cpu9:2097195)Queued:1 2021-04-26T00:45:48.759Z cpu9:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1. 2021-04-26T00:45:48.759Z cpu9:2100597)LVM: 6192: Could not open device mpx.vmhba32:C0:T0:L0:7, vol [60724578-09db3a26-f7cb-000af7abb22c, 60724578-09db3a26-f7cb-000af7abb22c, 1]: Timeout 2021-04-26T00:45:48.760Z cpu12:2100597)Vol3: 2129: Could not open device 'mpx.vmhba32:C0:T0:L0:7' for volume open: Not found 2021-04-26T00:45:48.760Z cpu12:2100597)Vol3: 4339: Failed to get object 28 type 1 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 0 gen 0 :Not found 2021-04-26T00:45:48.760Z cpu12:2100597)WARNING: Fil3: 1534: Failed to reserve volume f533 28 1 6072457a 332348ce a005c7a 2cb2abf7 0 0 0 0 0 0 0 2021-04-26T00:45:48.760Z cpu12:2100597)Vol3: 4339: Failed to get object 28 type 2 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 4 gen 1 :Not found 2021-04-26T00:45:48.796Z cpu11:2100351 opID=6e7a8f17)World: 11986: VC opID HB-SpecSync-host-15@1251066-79bed989-9b-ec66 maps to vmkernel opID 6e7a8f17 2021-04-26T00:45:48.796Z cpu11:2100351 opID=6e7a8f17)Config: 716: "HostLocalSwapDirEnabled" = 0, Old Value: 0, (Status: 0x0) 2021-04-26T00:45:52.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3657480) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:45:52.486Z cpu8:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:46:02.490Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3639480) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:46:02.490Z cpu15:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:46:04.485Z cpu4:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 7 times 2021-04-26T00:46:21.981Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a7c8 CmdSN 0x1e913 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:46:21.981Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c19fc0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:46:21.981Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389269a7c8 CmdSN 0x1e913 2021-04-26T00:46:21.981Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:46:21.981Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a7c8 CmdSN 0x1e913 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:46:21.981Z cpu11:2097360)Queued:2 2021-04-26T00:46:24.489Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e914 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:46:24.489Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:46:24.489Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e914 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:46:24.489Z cpu11:2097360)Queued:2 2021-04-26T00:46:24.493Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d00ec0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:46:28.761Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x9e, cmdId.initiator=0x45389639a3a8 CmdSN 0x1e915 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:46:28.761Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x9e, cmdId.initiator=0x45389639a3a8 CmdSN 0x1e915 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:46:28.761Z cpu8:2097360)Queued:1 2021-04-26T00:46:32.490Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0cad2c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:46:32.490Z cpu11:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:46:43.489Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3630880) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:46:43.490Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:47:01.981Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a8a8 CmdSN 0x1e91a from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:47:01.981Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:47:01.981Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a8a8 CmdSN 0x1e91a from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:47:01.981Z cpu11:2097360)Queued:1 2021-04-26T00:47:04.487Z cpu5:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times 2021-04-26T00:47:08.761Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e91f from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:47:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x25 (0x45d8c0d414c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:47:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a458 CmdSN 0x1e91f 2021-04-26T00:47:08.761Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:47:08.761Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e91f from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:47:08.761Z cpu11:2097360)Queued:1 2021-04-26T00:47:12.486Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3694d80) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:47:12.486Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:47:23.486Z cpu1:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3648480) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:47:23.487Z cpu8:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:47:41.982Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e1 from world 2100065 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:47:41.982Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:47:41.982Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e1 from world 2100065 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:47:41.983Z cpu11:2097360)Queued:1 2021-04-26T00:47:48.761Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e920 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:47:48.761Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:47:48.761Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e920 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:47:48.761Z cpu11:2097360)Queued:1 2021-04-26T00:47:52.488Z cpu5:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e368e480) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:47:52.488Z cpu11:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:48:03.489Z cpu5:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0cd57c0) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:48:03.489Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:48:04.487Z cpu7:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times 2021-04-26T00:48:21.985Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e2 from world 2100065 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:48:21.985Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x85 (0x45d8c0c19fc0, 2100065) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:48:21.985Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x4305118ff740 CmdSN 0x5e2 2021-04-26T00:48:21.985Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:48:21.985Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e2 from world 2100065 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:48:21.985Z cpu11:2097360)Queued:1 2021-04-26T00:48:21.992Z cpu10:2097746)ScsiDeviceIO: 4298: Cmd(0x45d8c0c19fc0) 0x85, CmdSN 0x5e5 from world 2100065 to dev "naa.61866da0a3e72500280513c4a60ad036" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0 2021-04-26T00:48:28.761Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e921 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:48:28.761Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:48:28.761Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e921 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:48:28.761Z cpu9:2097360)Queued:0 2021-04-26T00:48:32.489Z cpu7:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:48:32.489Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:48:43.488Z cpu7:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3711880) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:48:43.488Z cpu10:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:49:04.487Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times 2021-04-26T00:49:08.761Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x28, cmdId.initiator=0x430511941d80 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:49:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8c0d414c0, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:49:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511941d80 CmdSN 0x1 2021-04-26T00:49:08.761Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:49:08.762Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x28, cmdId.initiator=0x430511941d80 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:49:08.762Z cpu11:2097360)Queued:0 2021-04-26T00:49:08.762Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1. 2021-04-26T00:49:19.487Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c497c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:49:19.487Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:49:48.762Z cpu8:2097194)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x9e, cmdId.initiator=0x45389639a3a8 CmdSN 0x1e930 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:1 2021-04-26T00:49:48.762Z cpu8:2097194)Queued:0 2021-04-26T00:49:48.762Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c023c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e92f from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:49:48.762Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:49:48.762Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c023c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e92f from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:49:48.762Z cpu8:2097360)Queued:1 2021-04-26T00:49:48.762Z cpu11:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure 2021-04-26T00:49:59.487Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e363da80) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:49:59.488Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:50:04.486Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times 2021-04-26T00:50:14.491Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3734880) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:50:14.491Z cpu2:2097361)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x0 (0x45d8e3734880, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:50:14.491Z cpu2:2097361)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x45389f39bc58 CmdSN 0x0 2021-04-26T00:50:14.491Z cpu9:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:50:28.762Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e931 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:50:28.762Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:50:28.762Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e931 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:50:28.762Z cpu9:2097360)Queued:1 2021-04-26T00:50:39.487Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e369bb80) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:50:39.487Z cpu8:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:50:44.490Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e936 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:50:44.490Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e36dfd80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:50:44.490Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:50:44.490Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e936 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:50:44.490Z cpu9:2097360)Queued:1 2021-04-26T00:50:44.490Z cpu4:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout 2021-04-26T00:50:55.489Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c78fc0) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:50:55.489Z cpu8:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:51:04.486Z cpu0:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 5 times 2021-04-26T00:51:08.762Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e937 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:51:08.762Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0d37ec0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:51:08.762Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a398 CmdSN 0x1e937 2021-04-26T00:51:08.762Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:51:08.762Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e937 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:51:08.762Z cpu9:2097360)Queued:1 2021-04-26T00:51:19.488Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d658c0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2. 2021-04-26T00:51:19.488Z cpu15:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:51:24.491Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e36dfd80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:51:24.492Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e938 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:51:24.492Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:51:24.492Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e938 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:51:24.492Z cpu9:2097360)Queued:1 2021-04-26T00:51:35.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c3e8c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:51:35.487Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:51:48.764Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e939 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:51:48.764Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:51:48.764Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e939 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:51:48.764Z cpu8:2097360)Queued:0 2021-04-26T00:51:59.489Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d6fdc0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1. 2021-04-26T00:51:59.489Z cpu10:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry 2021-04-26T00:52:04.487Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times 2021-04-26T00:52:28.764Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0. 2021-04-26T00:52:28.764Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8c0d37ec0, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: 2021-04-26T00:52:28.764Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511963440 CmdSN 0x1 2021-04-26T00:52:28.764Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update... 2021-04-26T00:52:28.764Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:1 2021-04-26T00:52:28.764Z cpu8:2097360)Queued:0 2021-04-26T00:52:28.764Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1. 2021-04-26T00:52:28.764Z cpu8:2100597)LVM: 6192: Could not open device mpx.vmhba32:C0:T0:L0:7, vol [60724578-09db3a26-f7cb-000af7abb22c, 60724578-09db3a26-f7cb-000af7abb22c, 1]: Timeout 2021-04-26T00:52:28.765Z cpu8:2100597)Vol3: 2129: Could not open device 'mpx.vmhba32:C0:T0:L0:7' for volume open: Not found 2021-04-26T00:52:28.765Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 1 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 0 gen 0 :Not found 2021-04-26T00:52:28.765Z cpu8:2100597)WARNING: Fil3: 1534: Failed to reserve volume f533 28 1 6072457a 332348ce a005c7a 2cb2abf7 0 0 0 0 0 0 0 2021-04-26T00:52:28.765Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 2 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 4 gen 1 :Not found
2021-04-26T00:35:44.485Z cpu12:2097360)Queued:12021-04-26T00:35:44.485Z cpu2:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout2021-04-26T00:35:44.486Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c631c0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:35:48.735Z cpu12:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0dd6ac0) 0x28, cmdId.initiator=0x4305118d2080 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:35:48.735Z cpu12:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:35:48.735Z cpu12:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0dd6ac0) 0x28, cmdId.initiator=0x4305118d2080 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:35:48.735Z cpu12:2097360)Queued:12021-04-26T00:35:48.736Z cpu9:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1.2021-04-26T00:35:55.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c14fc0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:35:55.487Z cpu9:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:36:04.486Z cpu4:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times2021-04-26T00:36:06.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c48dc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:36:06.486Z cpu3:2097361)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x0 (0x45d8c0c48dc0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:36:06.486Z cpu3:2097361)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x453891e1bc58 CmdSN 0x02021-04-26T00:36:06.486Z cpu11:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:36:24.485Z cpu12:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e2127f40) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8d9 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:36:24.485Z cpu12:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:36:24.485Z cpu12:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e2127f40) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8d9 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:36:24.485Z cpu12:2097360)Queued:22021-04-26T00:36:24.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c631c0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:36:28.736Z cpu12:2097198)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x9e, cmdId.initiator=0x45389639a3f8 CmdSN 0x1e8db from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:12021-04-26T00:36:28.736Z cpu12:2097198)Queued:02021-04-26T00:36:28.737Z cpu12:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0cc62c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8da from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:36:28.738Z cpu12:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:36:28.738Z cpu12:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0cc62c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8da from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:36:28.738Z cpu12:2097360)Queued:12021-04-26T00:36:28.738Z cpu10:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure2021-04-26T00:36:34.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37e7480) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:36:34.486Z cpu10:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:36:45.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d112c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:36:45.485Z cpu9:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:37:04.485Z cpu4:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 5 times2021-04-26T00:37:08.739Z cpu13:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x25, cmdId.initiator=0x45389639a4a8 CmdSN 0x1e8e0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:37:08.739Z cpu13:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x25 (0x45d8e375f580, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:37:08.739Z cpu13:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a4a8 CmdSN 0x1e8e02021-04-26T00:37:08.739Z cpu13:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:37:08.739Z cpu13:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x25, cmdId.initiator=0x45389639a4a8 CmdSN 0x1e8e0 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:37:08.739Z cpu13:2097360)Queued:02021-04-26T00:37:19.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3721780) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:37:19.485Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:37:48.740Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a3e8 CmdSN 0x1e8e3 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:37:48.740Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:37:48.740Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a3e8 CmdSN 0x1e8e3 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:37:48.740Z cpu10:2097360)Queued:02021-04-26T00:37:59.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c14fc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:37:59.486Z cpu8:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:38:04.487Z cpu3:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times2021-04-26T00:38:28.741Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a4c8 CmdSN 0x1e8e4 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:38:28.741Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8e375f580, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:38:28.741Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a4c8 CmdSN 0x1e8e42021-04-26T00:38:28.741Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:38:28.741Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x1a, cmdId.initiator=0x45389639a4c8 CmdSN 0x1e8e4 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:38:28.741Z cpu10:2097360)Queued:02021-04-26T00:38:39.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e368cb80) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:38:39.486Z cpu10:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:39:04.485Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 1 times2021-04-26T00:39:08.744Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8e375f580) 0x28, cmdId.initiator=0x430511976f00 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:39:08.744Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8e375f580, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:39:08.744Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511976f00 CmdSN 0x12021-04-26T00:39:08.744Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:39:08.744Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8e375f580) 0x28, cmdId.initiator=0x430511976f00 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:39:08.744Z cpu9:2097360)Queued:02021-04-26T00:39:08.744Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1.2021-04-26T00:39:08.744Z cpu8:2100597)LVM: 6192: Could not open device mpx.vmhba32:C0:T0:L0:7, vol [60724578-09db3a26-f7cb-000af7abb22c, 60724578-09db3a26-f7cb-000af7abb22c, 1]: Timeout2021-04-26T00:39:08.745Z cpu8:2100597)Vol3: 2129: Could not open device 'mpx.vmhba32:C0:T0:L0:7' for volume open: Not found2021-04-26T00:39:08.745Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 1 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 0 gen 0 :Not found2021-04-26T00:39:08.745Z cpu8:2100597)WARNING: Fil3: 1534: Failed to reserve volume f533 28 1 6072457a 332348ce a005c7a 2cb2abf7 0 0 0 0 0 0 02021-04-26T00:39:08.745Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 2 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 4 gen 1 :Not found2021-04-26T00:39:19.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c5a5c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:39:19.486Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:39:48.744Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0ca50c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8e7 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:39:48.744Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:39:48.744Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0ca50c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8e7 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:39:48.744Z cpu9:2097360)Queued:12021-04-26T00:39:48.744Z cpu8:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure2021-04-26T00:39:48.748Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x9e, cmdId.initiator=0x45389639a458 CmdSN 0x1e8e8 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:39:48.748Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x9e, cmdId.initiator=0x45389639a458 CmdSN 0x1e8e8 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:39:48.748Z cpu9:2097360)Queued:02021-04-26T00:39:59.485Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d126c0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:39:59.485Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:40:04.485Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 4 times2021-04-26T00:40:14.486Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9d3c0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:40:14.486Z cpu6:2097361)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x0 (0x45d8c0c9d3c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:40:14.486Z cpu6:2097361)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x453885e1bc58 CmdSN 0x02021-04-26T00:40:14.486Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:40:28.750Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8e9 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:40:28.750Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:40:28.750Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8e9 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:40:28.750Z cpu10:2097360)Queued:12021-04-26T00:40:39.489Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c7e4c0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:40:39.489Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:40:44.488Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37d7f80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:40:44.489Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e8ee from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:40:44.489Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:40:44.489Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e8ee from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:40:44.489Z cpu8:2097360)Queued:12021-04-26T00:40:44.489Z cpu2:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout2021-04-26T00:40:55.486Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3660580) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:40:55.486Z cpu9:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:41:04.485Z cpu6:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 5 times2021-04-26T00:41:08.750Z cpu14:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8ef from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:41:08.750Z cpu14:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c9c4c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:41:08.750Z cpu14:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a448 CmdSN 0x1e8ef2021-04-26T00:41:08.750Z cpu14:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:41:08.750Z cpu14:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8ef from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:41:08.750Z cpu14:2097360)Queued:12021-04-26T00:41:19.487Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37a4680) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:41:19.487Z cpu8:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:41:24.488Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37d7f80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:41:24.490Z cpu14:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8f0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:41:24.490Z cpu14:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:41:24.490Z cpu14:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e6fafec0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e8f0 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:41:24.490Z cpu14:2097360)Queued:12021-04-26T00:41:35.488Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37aaa80) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:41:35.488Z cpu14:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:41:48.750Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e8f1 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:41:48.750Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:41:48.751Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e8f1 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:41:48.751Z cpu8:2097360)Queued:02021-04-26T00:41:59.485Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c4bfc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:41:59.485Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:42:04.485Z cpu6:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times2021-04-26T00:42:28.751Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:42:28.751Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8c0c9c4c0, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:42:28.751Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511963440 CmdSN 0x12021-04-26T00:42:28.751Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:42:28.751Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:42:28.751Z cpu9:2097360)Queued:02021-04-26T00:42:28.751Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1.2021-04-26T00:42:39.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d329c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:42:39.487Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:43:04.487Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 1 times2021-04-26T00:43:08.752Z cpu8:2097194)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x9e, cmdId.initiator=0x45389639a458 CmdSN 0x1e8f9 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:12021-04-26T00:43:08.752Z cpu8:2097194)Queued:02021-04-26T00:43:10.485Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c9c4c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8f8 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:43:10.486Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x35 (0x45d8c0c9c4c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:43:10.486Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45388621be18 CmdSN 0x1e8f82021-04-26T00:43:10.486Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:43:10.486Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c9c4c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e8f8 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:43:10.486Z cpu9:2097360)Queued:12021-04-26T00:43:10.486Z cpu8:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure2021-04-26T00:43:20.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e372a380) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:43:20.486Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:43:48.755Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c2c1c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8fa from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:43:48.755Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:43:48.755Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x25, cmdId.initiator=0x45389639a508 CmdSN 0x1e8fa from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:43:48.755Z cpu9:2097360)Queued:02021-04-26T00:43:59.487Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e368da80) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:43:59.487Z cpu10:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:44:04.487Z cpu5:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times2021-04-26T00:44:28.757Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8fd from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:44:28.757Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c2c1c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:44:28.757Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a448 CmdSN 0x1e8fd2021-04-26T00:44:28.757Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:44:28.757Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a448 CmdSN 0x1e8fd from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:44:28.757Z cpu8:2097360)Queued:12021-04-26T00:44:28.759Z cpu7:2101729)ALERT: hostd performance has degraded due to high system latency2021-04-26T00:44:39.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e377b280) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:44:39.486Z cpu8:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:45:01.981Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x9e, cmdId.initiator=0x45389269a7d8 CmdSN 0x1e909 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:45:01.981Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:45:01.981Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x9e, cmdId.initiator=0x45389269a7d8 CmdSN 0x1e909 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:45:01.981Z cpu10:2097360)Queued:12021-04-26T00:45:04.486Z cpu2:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 2 times2021-04-26T00:45:08.757Z cpu10:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e90a from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:45:08.757Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c2c1c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:45:08.757Z cpu10:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a528 CmdSN 0x1e90a2021-04-26T00:45:08.757Z cpu10:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:45:08.757Z cpu10:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x1a, cmdId.initiator=0x45389639a528 CmdSN 0x1e90a from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:45:08.757Z cpu10:2097360)Queued:22021-04-26T00:45:12.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e37ece80) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:45:12.487Z cpu11:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:45:23.490Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3626380) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:45:23.490Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:45:41.981Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x25, cmdId.initiator=0x45389269a888 CmdSN 0x1e90d from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:45:41.981Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:45:41.981Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x25, cmdId.initiator=0x45389269a888 CmdSN 0x1e90d from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:45:41.981Z cpu11:2097360)Queued:22021-04-26T00:45:44.488Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e912 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:45:44.488Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:45:44.488Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e912 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:45:44.488Z cpu11:2097360)Queued:22021-04-26T00:45:44.488Z cpu2:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout2021-04-26T00:45:44.491Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d00ec0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:45:48.759Z cpu9:2097195)ScsiDeviceIO: 4315: Cmd(0x45d8c0c2c1c0) 0x28, cmdId.initiator=0x4305119491c0 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:12021-04-26T00:45:48.759Z cpu9:2097195)Queued:12021-04-26T00:45:48.759Z cpu9:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1.2021-04-26T00:45:48.759Z cpu9:2100597)LVM: 6192: Could not open device mpx.vmhba32:C0:T0:L0:7, vol [60724578-09db3a26-f7cb-000af7abb22c, 60724578-09db3a26-f7cb-000af7abb22c, 1]: Timeout2021-04-26T00:45:48.760Z cpu12:2100597)Vol3: 2129: Could not open device 'mpx.vmhba32:C0:T0:L0:7' for volume open: Not found2021-04-26T00:45:48.760Z cpu12:2100597)Vol3: 4339: Failed to get object 28 type 1 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 0 gen 0 :Not found2021-04-26T00:45:48.760Z cpu12:2100597)WARNING: Fil3: 1534: Failed to reserve volume f533 28 1 6072457a 332348ce a005c7a 2cb2abf7 0 0 0 0 0 0 02021-04-26T00:45:48.760Z cpu12:2100597)Vol3: 4339: Failed to get object 28 type 2 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 4 gen 1 :Not found2021-04-26T00:45:48.796Z cpu11:2100351 opID=6e7a8f17)World: 11986: VC opID HB-SpecSync-host-15@1251066-79bed989-9b-ec66 maps to vmkernel opID 6e7a8f172021-04-26T00:45:48.796Z cpu11:2100351 opID=6e7a8f17)Config: 716: "HostLocalSwapDirEnabled" = 0, Old Value: 0, (Status: 0x0)2021-04-26T00:45:52.486Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3657480) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:45:52.486Z cpu8:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:46:02.490Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3639480) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:46:02.490Z cpu15:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:46:04.485Z cpu4:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 7 times2021-04-26T00:46:21.981Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a7c8 CmdSN 0x1e913 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:46:21.981Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0c19fc0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:46:21.981Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389269a7c8 CmdSN 0x1e9132021-04-26T00:46:21.981Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:46:21.981Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a7c8 CmdSN 0x1e913 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:46:21.981Z cpu11:2097360)Queued:22021-04-26T00:46:24.489Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e914 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:46:24.489Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:46:24.489Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e71efac0) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e914 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:46:24.489Z cpu11:2097360)Queued:22021-04-26T00:46:24.493Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d00ec0) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:46:28.761Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x9e, cmdId.initiator=0x45389639a3a8 CmdSN 0x1e915 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:46:28.761Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x9e, cmdId.initiator=0x45389639a3a8 CmdSN 0x1e915 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:46:28.761Z cpu8:2097360)Queued:12021-04-26T00:46:32.490Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0cad2c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:46:32.490Z cpu11:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:46:43.489Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3630880) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:46:43.490Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:47:01.981Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a8a8 CmdSN 0x1e91a from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:47:01.981Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:47:01.981Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x1a, cmdId.initiator=0x45389269a8a8 CmdSN 0x1e91a from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:47:01.981Z cpu11:2097360)Queued:12021-04-26T00:47:04.487Z cpu5:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times2021-04-26T00:47:08.761Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e91f from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:47:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x25 (0x45d8c0d414c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:47:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a458 CmdSN 0x1e91f2021-04-26T00:47:08.761Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:47:08.761Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e91f from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:47:08.761Z cpu11:2097360)Queued:12021-04-26T00:47:12.486Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3694d80) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:47:12.486Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:47:23.486Z cpu1:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3648480) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:47:23.487Z cpu8:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:47:41.982Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e1 from world 2100065 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:47:41.982Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:47:41.982Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e1 from world 2100065 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:47:41.983Z cpu11:2097360)Queued:12021-04-26T00:47:48.761Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e920 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:47:48.761Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:47:48.761Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e920 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:47:48.761Z cpu11:2097360)Queued:12021-04-26T00:47:52.488Z cpu5:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e368e480) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:47:52.488Z cpu11:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:48:03.489Z cpu5:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0cd57c0) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:48:03.489Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:48:04.487Z cpu7:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times2021-04-26T00:48:21.985Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e2 from world 2100065 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:48:21.985Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x85 (0x45d8c0c19fc0, 2100065) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:48:21.985Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x4305118ff740 CmdSN 0x5e22021-04-26T00:48:21.985Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:48:21.985Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c19fc0) 0x85, cmdId.initiator=0x4305118ff740 CmdSN 0x5e2 from world 2100065 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:48:21.985Z cpu11:2097360)Queued:12021-04-26T00:48:21.992Z cpu10:2097746)ScsiDeviceIO: 4298: Cmd(0x45d8c0c19fc0) 0x85, CmdSN 0x5e5 from world 2100065 to dev "naa.61866da0a3e72500280513c4a60ad036" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x02021-04-26T00:48:28.761Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e921 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:48:28.761Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:48:28.761Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e921 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:48:28.761Z cpu9:2097360)Queued:02021-04-26T00:48:32.489Z cpu7:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c19fc0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:48:32.489Z cpu9:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:48:43.488Z cpu7:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3711880) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:48:43.488Z cpu10:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:49:04.487Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times2021-04-26T00:49:08.761Z cpu11:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d414c0) 0x28, cmdId.initiator=0x430511941d80 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:49:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8c0d414c0, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:49:08.761Z cpu11:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511941d80 CmdSN 0x12021-04-26T00:49:08.761Z cpu11:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:49:08.762Z cpu11:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d414c0) 0x28, cmdId.initiator=0x430511941d80 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:49:08.762Z cpu11:2097360)Queued:02021-04-26T00:49:08.762Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1.2021-04-26T00:49:19.487Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c497c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:49:19.487Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:49:48.762Z cpu8:2097194)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x9e, cmdId.initiator=0x45389639a3a8 CmdSN 0x1e930 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from device layer. Cmd count Active:12021-04-26T00:49:48.762Z cpu8:2097194)Queued:02021-04-26T00:49:48.762Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c023c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e92f from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:49:48.762Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:49:48.762Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0c023c0) 0x35, cmdId.initiator=0x45388621be18 CmdSN 0x1e92f from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:49:48.762Z cpu8:2097360)Queued:12021-04-26T00:49:48.762Z cpu11:2097348)ScsiDevice: 979: Could not flush cache of local device mpx.vmhba32:C0:T0:L0. Failure2021-04-26T00:49:59.487Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e363da80) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:49:59.488Z cpu15:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:50:04.486Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 3 times2021-04-26T00:50:14.491Z cpu2:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e3734880) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:50:14.491Z cpu2:2097361)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x0 (0x45d8e3734880, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:50:14.491Z cpu2:2097361)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x45389f39bc58 CmdSN 0x02021-04-26T00:50:14.491Z cpu9:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:50:28.762Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e931 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:50:28.762Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:50:28.762Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x25, cmdId.initiator=0x45389639a458 CmdSN 0x1e931 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:50:28.762Z cpu9:2097360)Queued:12021-04-26T00:50:39.487Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e369bb80) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:50:39.487Z cpu8:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:50:44.490Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e936 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:50:44.490Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e36dfd80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:50:44.490Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:50:44.490Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcb8 CmdSN 0x1e936 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:50:44.490Z cpu9:2097360)Queued:12021-04-26T00:50:44.490Z cpu4:2097341)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L0 failed with error Timeout2021-04-26T00:50:55.489Z cpu6:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c78fc0) 0x0, cmdId.initiator=0x4538a099bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:50:55.489Z cpu8:2111251)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:51:04.486Z cpu0:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 5 times2021-04-26T00:51:08.762Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e937 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:51:08.762Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x1a (0x45d8c0d37ec0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:51:08.762Z cpu9:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x45389639a398 CmdSN 0x1e9372021-04-26T00:51:08.762Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:51:08.762Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a398 CmdSN 0x1e937 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:51:08.762Z cpu9:2097360)Queued:12021-04-26T00:51:19.488Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d658c0) 0x0, cmdId.initiator=0x453891e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:2.2021-04-26T00:51:19.488Z cpu15:2111249)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:51:24.491Z cpu3:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8e36dfd80) 0xa0, cmdId.initiator=0x453885d9b8f8 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:51:24.492Z cpu9:2097360)ScsiPath: 8058: Cancelled Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e938 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:51:24.492Z cpu9:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:51:24.492Z cpu9:2097360)ScsiDeviceIO: 4315: Cmd(0x45b8e202d040) 0x12, cmdId.initiator=0x453885e9bcc8 CmdSN 0x1e938 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:51:24.492Z cpu9:2097360)Queued:12021-04-26T00:51:35.487Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0c3e8c0) 0x0, cmdId.initiator=0x45389f39bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:51:35.487Z cpu8:2111250)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:51:48.764Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e939 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:51:48.764Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:51:48.764Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x1a, cmdId.initiator=0x45389639a478 CmdSN 0x1e939 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:51:48.764Z cpu8:2097360)Queued:02021-04-26T00:51:59.489Z cpu0:2097361)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d6fdc0) 0x0, cmdId.initiator=0x453885e1bc58 CmdSN 0x0 from world 0 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:1.2021-04-26T00:51:59.489Z cpu10:2097340)VMW_SATP_LOCAL: satp_local_updatePath:856: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry2021-04-26T00:52:04.487Z cpu1:2097624)NMP: nmp_ResetDeviceLogThrottling:3782: last error status from device mpx.vmhba32:C0:T0:L0 repeated 6 times2021-04-26T00:52:28.764Z cpu8:2097360)ScsiPath: 8058: Cancelled Cmd(0x45d8c0d37ec0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to path "vmhba32:C0:T0:L0". Cmd count Active:0 Queued:0.2021-04-26T00:52:28.764Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3861: Cmd 0x28 (0x45d8c0d37ec0, 2100012) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed:2021-04-26T00:52:28.764Z cpu8:2097360)NMP: nmp_ThrottleLogForDevice:3869: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x430511963440 CmdSN 0x12021-04-26T00:52:28.764Z cpu8:2097360)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...2021-04-26T00:52:28.764Z cpu8:2097360)ScsiDeviceIO: 4315: Cmd(0x45d8c0d37ec0) 0x28, cmdId.initiator=0x430511963440 CmdSN 0x1 from world 2100012 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x5 D:0x0 P:0x0 Cancelled from path layer. Cmd count Active:12021-04-26T00:52:28.764Z cpu8:2097360)Queued:02021-04-26T00:52:28.764Z cpu8:2100597)LVM: 6817: Forcing APD unregistration of devID 60724579-9ffb2d25-6912-000af7abb22c in state 1.2021-04-26T00:52:28.764Z cpu8:2100597)LVM: 6192: Could not open device mpx.vmhba32:C0:T0:L0:7, vol [60724578-09db3a26-f7cb-000af7abb22c, 60724578-09db3a26-f7cb-000af7abb22c, 1]: Timeout2021-04-26T00:52:28.765Z cpu8:2100597)Vol3: 2129: Could not open device 'mpx.vmhba32:C0:T0:L0:7' for volume open: Not found2021-04-26T00:52:28.765Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 1 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 0 gen 0 :Not found2021-04-26T00:52:28.765Z cpu8:2100597)WARNING: Fil3: 1534: Failed to reserve volume f533 28 1 6072457a 332348ce a005c7a 2cb2abf7 0 0 0 0 0 0 02021-04-26T00:52:28.765Z cpu8:2100597)Vol3: 4339: Failed to get object 28 type 2 uuid 6072457a-332348ce-5c7a-000af7abb22c FD 4 gen 1 :Not found