In the last couple days my Esxi server has started getting PSOD at around 5am almost every day (thur, fri, sun).. I have attached the screenshot of PSOD....I tried to attach vmkernal-zdump but it is too large. I have added the lines below that I found when searching error or failed and wondering if I am getting hacked because the IP address listed is one I am not familiar with.
Error reading from pending connection: Connection closed by remote host, possibly due to timeout
2016-04-12T00:34:40.829Z cpu4:8770)MigrateNet:
NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124038cc000, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T06:53:58.264Z cpu11:6020559)ScsiDeviceIO: 2331: Cmd(0x4124038cc000) 0x85, CmdSN 0x8f2868f from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T06:53:58.264Z cpu11:6020559)ScsiDeviceIO: 2331: Cmd(0x4124038cc000) 0x4d, CmdSN 0x8f28690 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T06:53:58.264Z cpu11:6020559)ScsiDeviceIO: 2331: Cmd(0x4124038cc000) 0x1a, CmdSN 0x8f28691 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T07:23:58.315Z cpu8:6020553)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124003b9100, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T07:23:58.315Z cpu8:6020553)ScsiDeviceIO: 2331: Cmd(0x4124003b9100) 0x85, CmdSN 0x8f2b7ca from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T07:23:58.315Z cpu8:6020553)ScsiDeviceIO: 2331: Cmd(0x4124003b9100) 0x4d, CmdSN 0x8f2b7cb from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T07:23:58.315Z cpu8:6020553)ScsiDeviceIO: 2331: Cmd(0x4124003b9100) 0x1a, CmdSN 0x8f2b7cc from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T07:47:31.655Z cpu4:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:47:31.655Z cpu4:8770)MigrateNet: vm 8770: 2147: dataSocket 0x410004569c30 receive buffer size is 562480
2016-04-06T07:47:42.655Z cpu4:8770)Migrate: 208: Error reading from pending connection: Connection closed by remote host, possibly due to timeout
2016-04-06T07:47:42.756Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:47:42.756Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x4100046c8a40 receive buffer size is 562480
2016-04-06T07:47:47.704Z cpu2:8770)Migrate: 208: Error reading from pending connection: Connection closed by remote host, possibly due to timeout
2016-04-06T07:47:47.806Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:47:47.806Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x4100047430b0 receive buffer size is 562480
2016-04-06T07:47:52.751Z cpu2:8770)Migrate: 208: Error reading from pending connection: Connection closed by remote host, possibly due to timeout
2016-04-06T07:48:04.483Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:04.483Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x410004569c30 receive buffer size is 562480
2016-04-06T07:48:04.483Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:04.483Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x4100046c8a40 receive buffer size is 562480
2016-04-06T07:48:04.625Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:04.625Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x4100047430b0 receive buffer size is 562480
[7m2016-04-06T07:48:05.051Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 1230262351. Expecting message of type INIT (0). [0m
[7m2016-04-06T07:48:05.153Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 1414745936. Expecting message of type INIT (0). [0m
2016-04-06T07:48:05.494Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:05.494Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x410004569c30 receive buffer size is 562480
[7m2016-04-06T07:48:05.494Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 16843542. Expecting message of type INIT (0). [0m
2016-04-06T07:48:05.596Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:05.596Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x4100046c8a40 receive buffer size is 562480
[7m2016-04-06T07:48:05.596Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 16843542. Expecting message of type INIT (0). [0m
2016-04-06T07:48:05.913Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:05.913Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x410004569c30 receive buffer size is 562480
2016-04-06T07:48:05.915Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:05.915Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x4100046c8a40 receive buffer size is 562480
2016-04-06T07:48:05.968Z cpu2:8770)MigrateNet: vm 8770: 2077: Accepted connection from <45.33.3.175>
2016-04-06T07:48:05.968Z cpu2:8770)MigrateNet: vm 8770: 2147: dataSocket 0x41000473f540 receive buffer size is 562480
[7m2016-04-06T07:48:06.061Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 16843542. Expecting message of type INIT (0). [0m
[7m2016-04-06T07:48:06.164Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 16843542. Expecting message of type INIT (0). [0m
[7m2016-04-06T07:48:06.266Z cpu2:8770)WARNING: Migrate: 262: Invalid message type for new connection: 1347375696. Expecting message of type INIT (0). [0m
2016-04-06T07:48:25.595Z cpu2:8770)Migrate: 208: Error reading from pending connection: Connection closed by remote host, possibly due to timeout
2016-04-06T07:53:58.353Z cpu0:8216)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412403965d40, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T07:53:58.353Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412403965d40) 0x85, CmdSN 0x8f2e904 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T07:53:58.354Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412403965d40) 0x4d, CmdSN 0x8f2e905 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T07:53:58.354Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412403965d40) 0x1a, CmdSN 0x8f2e906 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T08:23:58.396Z cpu7:8199)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x41240335d5c0, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T08:23:58.396Z cpu7:8199)ScsiDeviceIO: 2331: Cmd(0x41240335d5c0) 0x85, CmdSN 0x8f31a3f from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T08:23:58.396Z cpu7:8199)ScsiDeviceIO: 2331: Cmd(0x41240335d5c0) 0x4d, CmdSN 0x8f31a40 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T08:23:58.396Z cpu7:8199)ScsiDeviceIO: 2331: Cmd(0x41240335d5c0) 0x1a, CmdSN 0x8f31a41 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T08:53:58.437Z cpu10:8202)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x41240039d680, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T08:53:58.437Z cpu10:8202)ScsiDeviceIO: 2331: Cmd(0x41240039d680) 0x85, CmdSN 0x8f34b79 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T08:53:58.437Z cpu10:8202)ScsiDeviceIO: 2331: Cmd(0x41240039d680) 0x4d, CmdSN 0x8f34b7a from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T08:53:58.437Z cpu10:8202)ScsiDeviceIO: 2331: Cmd(0x41240039d680) 0x1a, CmdSN 0x8f34b7b from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T09:23:58.482Z cpu0:8216)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124003aa2c0, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T09:23:58.482Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x4124003aa2c0) 0x85, CmdSN 0x8f37cb1 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T09:23:58.482Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x4124003aa2c0) 0x4d, CmdSN 0x8f37cb2 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T09:23:58.483Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x4124003aa2c0) 0x1a, CmdSN 0x8f37cb3 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T09:53:58.523Z cpu10:6020552)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x41240426b100, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T09:53:58.523Z cpu10:6020552)ScsiDeviceIO: 2331: Cmd(0x41240426b100) 0x85, CmdSN 0x8f3adec from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T09:53:58.523Z cpu10:6020552)ScsiDeviceIO: 2331: Cmd(0x41240426b100) 0x4d, CmdSN 0x8f3aded from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T09:53:58.524Z cpu10:6020552)ScsiDeviceIO: 2331: Cmd(0x41240426b100) 0x1a, CmdSN 0x8f3adee from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T10:23:58.575Z cpu9:8201)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412404343800, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T10:23:58.575Z cpu9:8201)ScsiDeviceIO: 2331: Cmd(0x412404343800) 0x85, CmdSN 0x8f3df27 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T10:23:58.575Z cpu9:8201)ScsiDeviceIO: 2331: Cmd(0x412404343800) 0x4d, CmdSN 0x8f3df28 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T10:23:58.575Z cpu9:8201)ScsiDeviceIO: 2331: Cmd(0x412404343800) 0x1a, CmdSN 0x8f3df29 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T10:53:58.620Z cpu8:8200)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124039fd340, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T10:53:58.620Z cpu8:8200)ScsiDeviceIO: 2331: Cmd(0x4124039fd340) 0x85, CmdSN 0x8f41060 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T10:53:58.620Z cpu8:8200)ScsiDeviceIO: 2331: Cmd(0x4124039fd340) 0x4d, CmdSN 0x8f41061 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T10:53:58.620Z cpu8:8200)ScsiDeviceIO: 2331: Cmd(0x4124039fd340) 0x1a, CmdSN 0x8f41062 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T11:23:58.669Z cpu0:8216)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124011b6900, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T11:23:58.669Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x4124011b6900) 0x85, CmdSN 0x8f4419a from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T11:23:58.669Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x4124011b6900) 0x4d, CmdSN 0x8f4419b from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T11:23:58.669Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x4124011b6900) 0x1a, CmdSN 0x8f4419c from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T11:53:58.721Z cpu4:8196)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x41240117a5c0, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T11:53:58.721Z cpu4:8196)ScsiDeviceIO: 2331: Cmd(0x41240117a5c0) 0x85, CmdSN 0x8f472d4 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T11:53:58.721Z cpu4:8196)ScsiDeviceIO: 2331: Cmd(0x41240117a5c0) 0x4d, CmdSN 0x8f472d5 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T11:53:58.721Z cpu4:8196)ScsiDeviceIO: 2331: Cmd(0x41240117a5c0) 0x1a, CmdSN 0x8f472d6 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T12:23:58.764Z cpu0:8216)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412404e93440, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T12:23:58.764Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412404e93440) 0x85, CmdSN 0x8f4a40f from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T12:23:58.764Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412404e93440) 0x4d, CmdSN 0x8f4a410 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T12:23:58.764Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412404e93440) 0x1a, CmdSN 0x8f4a411 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T12:53:58.807Z cpu5:8197)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412404336640, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T12:53:58.807Z cpu5:8197)ScsiDeviceIO: 2331: Cmd(0x412404336640) 0x85, CmdSN 0x8f4d549 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T12:53:58.807Z cpu5:8197)ScsiDeviceIO: 2331: Cmd(0x412404336640) 0x4d, CmdSN 0x8f4d54a from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T12:53:58.807Z cpu5:8197)ScsiDeviceIO: 2331: Cmd(0x412404336640) 0x1a, CmdSN 0x8f4d54b from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T13:23:58.851Z cpu0:8216)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412403a590c0, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T13:23:58.851Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412403a590c0) 0x85, CmdSN 0x8f506ae from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T13:23:58.851Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412403a590c0) 0x4d, CmdSN 0x8f506af from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T13:23:58.851Z cpu0:8216)ScsiDeviceIO: 2331: Cmd(0x412403a590c0) 0x1a, CmdSN 0x8f506b0 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T13:53:58.893Z cpu2:8194)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412403291cc0, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T13:53:58.893Z cpu2:8194)ScsiDeviceIO: 2331: Cmd(0x412403291cc0) 0x85, CmdSN 0x8f537e9 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T13:53:58.893Z cpu2:8194)ScsiDeviceIO: 2331: Cmd(0x412403291cc0) 0x4d, CmdSN 0x8f537ea from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T13:53:58.893Z cpu2:8194)ScsiDeviceIO: 2331: Cmd(0x412403291cc0) 0x1a, CmdSN 0x8f537eb from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T14:23:58.935Z cpu2:8194)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124003b3280, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T14:23:58.935Z cpu2:8194)ScsiDeviceIO: 2331: Cmd(0x4124003b3280) 0x85, CmdSN 0x8f5694e from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T14:23:58.935Z cpu2:8194)ScsiDeviceIO: 2331: Cmd(0x4124003b3280) 0x4d, CmdSN 0x8f5694f from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T14:23:58.935Z cpu2:8194)ScsiDeviceIO: 2331: Cmd(0x4124003b3280) 0x1a, CmdSN 0x8f56950 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T14:53:59.206Z cpu7:9598)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x41240472b900, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T14:53:59.206Z cpu7:9598)ScsiDeviceIO: 2331: Cmd(0x41240472b900) 0x85, CmdSN 0x8f59a89 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T14:53:59.218Z cpu7:9598)ScsiDeviceIO: 2331: Cmd(0x41240472b900) 0x4d, CmdSN 0x8f59a8a from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T14:53:59.228Z cpu7:9598)ScsiDeviceIO: 2331: Cmd(0x41240472b900) 0x1a, CmdSN 0x8f59a8b from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T15:23:59.270Z cpu10:8202)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412401f31c00, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T15:23:59.270Z cpu10:8202)ScsiDeviceIO: 2331: Cmd(0x412401f31c00) 0x85, CmdSN 0x8f5cbc4 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T15:23:59.270Z cpu10:8202)ScsiDeviceIO: 2331: Cmd(0x412401f31c00) 0x4d, CmdSN 0x8f5cbc5 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T15:23:59.270Z cpu10:8202)ScsiDeviceIO: 2331: Cmd(0x412401f31c00) 0x1a, CmdSN 0x8f5cbc6 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T15:53:59.607Z cpu1:8193)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x412403956400, 9487) to dev "naa.600605b0064632501a0becd41125abe2" on path "vmhba1:C2:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2016-04-06T15:53:59.607Z cpu1:8193)ScsiDeviceIO: 2331: Cmd(0x412403956400) 0x85, CmdSN 0x8f5fd06 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T15:53:59.653Z cpu1:8193)ScsiDeviceIO: 2331: Cmd(0x412403956400) 0x4d, CmdSN 0x8f5fd07 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2016-04-06T15:53:59.683Z cpu1:8193)ScsiDeviceIO: 2331: Cmd(0x412403956400) 0x1a, CmdSN 0x8f5fd08 from world 9487 to dev "naa.600605b0064632501a0becd41125abe2" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-04-06T16:23:59.722Z cpu4:6020549)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x85 (0x4124003
THANKS for any help you can give me!!!
Have you identified virtual machines with E1000e virtual network and replaced by VMXNET3 like described on the following KB: VMware KB: VMware ESXi 5.x host experiences a purple diagnostic screen mentioning E1000PollRxRing an...
I think you need to install ESXi 5.1 Update 3
ESXi 5.1 Update 3 Release Notes
Page fault might occur while using XvMotion feature on ESXi hosts
A page fault might occur when you enable all optimizations while using the XvMotion feature on ESXi hosts with a datastore of block size 1MB. A purple screen with a message similar to the following is displayed:
PSOD @BlueScreen: #PF Exception 14 in world 1000046563:helper14-2 IP 0x4180079313ab addr 0x4130045c3600
This issue is resolved in this release.
see similar question - ESXi 5.1 Update 1 crashing randomly with purple screen #PF Exception 14
Seems like you are experiencing this issue: VMware KB: VMware ESXi 5.x host experiences a purple diagnostic screen mentioning E1000PollRxRing an...
A workaround if you can't upgrade your host is replace the E1000e virtual NCI with VMXNET3.
Updating to update 3 did not work, server crashed again at 5:21am Any other ideas?
can you share PSOD screen image and attach vmkernel.log file?
what is the ESXi Build number now?
Have you identified virtual machines with E1000e virtual network and replaced by VMXNET3 like described on the following KB: VMware KB: VMware ESXi 5.x host experiences a purple diagnostic screen mentioning E1000PollRxRing an...
They all use them, only 2 vm's I have not replaced them since I thought the update fixed them and replacing with vmxnet3 was a work around if I could not upgrade.
I attached the psod to the original post and pasted in the log as it was too big. I can't figure out how to add another attachment.....or I would....
Please try replacing e1000 Ethernet to vmxnet3. However I have seen another KB with similar error where vmxnet3 was used. ( searched KBs again, but not able to find it now)
If this don't fix the issue, I would suggest you to create ticket with VMware and Hardware Vendor, Provide them dump file and they can point you to exact issue.
Ok, Just very crazy that this server has been 100% stable for the last 2 years and then this started happening almost every morning at 5am. So are we sure this is a hardware issue and not someone trying to hack me?
I am not looking forward to the time or cost working with vmware support and lenovo and want to make sure before I go that route.
Thanks!
I don't think it has anything with hack.
it should be related to some hardware or driver issues.
Is this correct. I have to delete the adapter on each vm and then add it again and choose vmxnet3 instead or e1000 and then assign my IP address in vm.
Changing the NIC's seemed to fix the issue. Found out I had a FTP job backing up to one of my servers at 5am and it recently got changed to active instead of passive, looks like this was causing it to crash, changed the nic and it stayed up and backup still completed.
THANKS FOR THE HELP!!!!!!