VMware Cloud Community
Noein
Enthusiast
Enthusiast
Jump to solution

Could not complete network copy for file /vmfs/volumes/*.vmdk

I have VMWare Essentials Plus with 2 servers in it. Yesterday i have reinstalled ESXi on one of them.

1) When i tryed to add host back to ESXi, vCenter told me that datastore from new ESXi host is allready exists in cluster

2) I found guide, unmounted, deleted and created new datastore on this ESXi host, so it was added to vCenter without problems.

3) I configured vMotion and start trying migrate VMs to this host and reciaved message: Could not complete network copy for file /vmfs/volumes/*.vmdk

when i reisntalled ESXi and added it to vCenter before, there are allways no problems, but this time, i migration dont whant to work.

- MTU on vSwitch where configured vMotion is 1500 on both hosts

- I checked with vmkping, each host can ping another

Servers connected through local network of company on one interface and sirectly one to another on second interface. Both interfaces are pingable with vmkping and have MTU 1500.

Please help me :smileyshocked:

1 Solution

Accepted Solutions
Nawals
Expert
Expert
Jump to solution

Make sure both ESXi with same build. this could be the one reason for failing migration. Please update the ESXi to make compliance.

NKS Please Mark Helpful/correct if my answer resolve your query.

View solution in original post

19 Replies
Nawals
Expert
Expert
Jump to solution

Can You check the network security policy on both host are same or different. If policy is not same it will not allow to migrate the VMs.

NKS Please Mark Helpful/correct if my answer resolve your query.
Noein
Enthusiast
Enthusiast
Jump to solution

On both hosts on switches that vMotion configured on, settings are same:

Promiscuous mode: Reject

MAC address changes: Accept

Forged transmits: Accept

P.S.

I have checked vmkping -d -s 8972 -I vmk1 my_host_ip, from each host to another, it results as "sendto() failed (Message too long)"

I have changed MTU on both switches to 9000, now i have active migration process of test VM, that started from 23% and stuck. So it is 23% for allready half of hour

Now vmkping -d -s 8972 -I vmk1 my_host_ip fro each host to another, results as normal ping

Reply
0 Kudos
Nawals
Expert
Expert
Jump to solution

Follow this KB VMware Knowledge Base

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
Alex_Romeo
Leadership
Leadership
Jump to solution

Reply
0 Kudos
Noein
Enthusiast
Enthusiast
Jump to solution

I started from this threads. No result Smiley Sad

Reply
0 Kudos
Noein
Enthusiast
Enthusiast
Jump to solution

I have edited config like here VMware Knowledge Base . Now i have error: An error occurred while communicating with the remote host.

Smiley Sad

Reply
0 Kudos
Nawals
Expert
Expert
Jump to solution

Yyou can revert the changes and restart the ESXi and check. If issue still persist please share the vmkernel logs...

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
Noein
Enthusiast
Enthusiast
Jump to solution

After all manipulations, i have folowing:

[root@esxi217:/var/log] esxcli network nic list

Name    PCI Device    Driver  Admin Status  Link Status  Speed  Duplex  MAC Address         MTU  Description

------  ------------  ------  ------------  -----------  -----  ------  -----------------  ----  -------------------------------------------------------------------

vmnic0  0000:04:00.0  ixgben  Up            Up            1000  Full    e4:11:5b:95:79:84  9000  Intel Corporation 82599EB 10 Gigabit Dual Port Backplane Connection

vmnic1  0000:04:00.1  ixgben  Up            Up            1000  Full    e4:11:5b:95:79:85  9000  Intel Corporation 82599EB 10 Gigabit Dual Port Backplane Connection

[root@esxi225:/var/log] esxcli network nic list

Name    PCI Device    Driver  Admin Status  Link Status  Speed  Duplex  MAC Address         MTU  Description

------  ------------  ------  ------------  -----------  -----  ------  -----------------  ----  --------------------------------------------------------

vmnic0  0000:04:00.0  ixgben  Up            Up            1000  Full    e4:11:5b:96:56:b8  9000  Intel(R) 82599 10 Gigabit Dual Port Backplane Connection

vmnic1  0000:04:00.1  ixgben  Up            Up            1000  Full    e4:11:5b:96:56:b9  9000  Intel(R) 82599 10 Gigabit Dual Port Backplane Connection

vmkping -d -s 8972 from each host to each results as 100% loss:

3 packets transmitted, 0 packets received, 100% packet loss

even on vmnic1, this interfaces connected directly withcable

vmkping -d -s 1450 from each host to each results as normal ping

vmkping -d -s 8972 from each host to itself results as normal ping

vmkernel logs taken just after i tryed to migrate VM

Is it possible to hide it under spoilers? i cannot find such element in redactor

host from where i trying to migrate VM:

[root@esxi217:/var/log] tail -n 100 vmkernel.log

2020-03-23T04:56:10.412Z cpu26:2391639)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T04:56:10.413Z cpu26:2391639)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T04:56:11.144Z cpu6:2393421)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T05:04:01.185Z cpu20:2129039)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1f2, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:04:01.185Z cpu20:2097600)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1f3, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:04:01.191Z cpu20:2129039)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1f5, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:04:01.191Z cpu20:2129039)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1f6, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:06:21.269Z cpu24:2393537)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:06:21.269Z cpu24:2393537)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:06:21.785Z cpu16:2395460)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T05:14:59.001Z cpu21:2101198)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x73ca3, worldId: 0x246381, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:15:34.840Z cpu17:2097543)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x73cd0, worldId: 0x246381, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:16:30.204Z cpu16:2395560)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:16:30.204Z cpu16:2395560)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:16:30.829Z cpu13:2397356)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T05:26:40.257Z cpu20:2397459)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:26:40.257Z cpu20:2397459)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:26:40.844Z cpu26:2399184)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T05:34:01.234Z cpu16:2101193)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1f8, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:34:01.234Z cpu16:2101193)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1f9, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:34:01.240Z cpu16:2101193)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1fb, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:34:01.240Z cpu16:2101193)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1fc, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:35:36.917Z cpu17:2098022)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x740fd, worldId: 0x22d549, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:36:51.104Z cpu22:2399284)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:36:51.106Z cpu22:2399284)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:36:51.766Z cpu6:2401067)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T05:47:01.219Z cpu17:2401167)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:47:01.219Z cpu17:2401167)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:47:01.825Z cpu16:2402926)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T05:52:49.209Z cpu16:2101940)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x74466, worldId: 0x246381, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:57:11.052Z cpu18:2403023)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:57:11.052Z cpu18:2403023)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T05:57:11.626Z cpu8:2404823)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T06:04:01.265Z cpu31:2101197)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1fe, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:04:01.265Z cpu31:2101197)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x1ff, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:04:01.270Z cpu31:2101197)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x201, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:04:01.271Z cpu31:2101197)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x202, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:07:22.181Z cpu26:2404941)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:07:22.181Z cpu26:2404941)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:07:22.863Z cpu5:2406876)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T06:17:32.007Z cpu31:2407003)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:17:32.007Z cpu31:2407003)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:17:32.699Z cpu19:2408768)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T06:25:03.687Z cpu21:2101365)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x74a81, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:27:41.607Z cpu18:2408870)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:27:41.608Z cpu18:2408870)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:27:42.313Z cpu14:2410676)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T06:34:01.296Z cpu16:2101365)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x204, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:34:01.296Z cpu16:2101365)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x205, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:34:01.301Z cpu16:2101365)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x207, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:34:01.301Z cpu16:2101365)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x208, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:37:51.973Z cpu24:2410794)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:37:51.973Z cpu24:2410794)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:37:52.695Z cpu12:2412562)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T06:48:02.266Z cpu18:2412662)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:48:02.266Z cpu18:2412662)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:48:02.932Z cpu21:2414516)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T06:51:37.151Z cpu21:2129044)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x74fa1, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:58:12.104Z cpu27:2414617)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:58:12.104Z cpu27:2414617)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T06:58:12.641Z cpu20:2416371)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:04:01.433Z cpu17:2101947)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x20a, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:04:01.433Z cpu17:2097600)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x20b, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:04:01.436Z cpu17:2101947)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x20d, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:04:01.437Z cpu17:2101947)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x20e, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:06:17.526Z cpu17:2101942)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x7526c, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:08:22.996Z cpu31:2416469)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:08:22.996Z cpu31:2416469)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:08:23.730Z cpu0:2418465)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:18:33.002Z cpu28:2418566)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:18:33.002Z cpu28:2418566)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:18:33.534Z cpu0:2420247)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:22:18.753Z cpu18:2101939)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x755a5, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:27:15.062Z cpu16:2101365)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x7589a, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:28:47.877Z cpu16:2420364)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:28:47.877Z cpu16:2420364)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:28:48.511Z cpu4:2422190)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:32:06.132Z cpu17:2102121)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x75b40, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:34:01.466Z cpu19:2097856)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x210, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:34:01.466Z cpu19:2097856)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x211, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:34:01.469Z cpu19:2101929)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x213, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:34:01.469Z cpu19:2101929)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x214, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:38:58.597Z cpu20:2422302)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:38:58.597Z cpu20:2422302)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:38:59.215Z cpu12:2424066)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:49:13.003Z cpu29:2424170)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:49:13.003Z cpu29:2424170)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:49:13.619Z cpu30:2425956)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:59:24.216Z cpu16:2426055)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:59:24.216Z cpu16:2426055)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T07:59:24.736Z cpu10:2427828)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T07:59:28.829Z cpu16:2129044)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x76208, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:04:01.492Z cpu17:2098022)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x216, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:04:01.492Z cpu17:2097600)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x217, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:04:01.498Z cpu17:2098022)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x219, worldId: 0x2009ad, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:04:01.498Z cpu17:2097600)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x21a, worldId: 0x2009ad, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:09:33.065Z cpu0:2427948)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T08:09:33.065Z cpu11:2427948)WARNING: UserSocketInet: 2266: python: waiters list not empty!

2020-03-23T08:09:33.652Z cpu22:2429917)WARNING: MemSchedAdmit: 1226: Group vsanperfsvc: Requested memory limit 0 KB insufficient to support effective reservation 10296 KB

2020-03-23T08:09:55.354Z cpu21:2406220)nhpsa: hpsa_vmkScsiCmdDone:6393: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x76475, worldId: 0x24b74c, Cmd: 0x93, ASC: 0x20, ASCQ: 0x0

host where i am trying to migrate VM:

[root@esxi225:/var/log] tail -n 100 vmkernel.log

2020-03-23T01:04:15.578Z cpu0:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T01:07:22.336Z cpu39:2097994)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x43, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T01:07:22.336Z cpu39:2097994)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x44, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T01:08:15.578Z cpu8:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T01:14:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T01:24:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T01:30:40.165Z cpu20:2097231)ScsiDeviceIO: 3435: Cmd(0x45a247c3d4c0) 0x1a, CmdSN 0xa84 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T01:34:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T01:37:22.613Z cpu39:2097994)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x48, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T01:37:22.613Z cpu39:2097723)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x49, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T01:38:15.578Z cpu27:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T01:44:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T01:54:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T02:04:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T02:07:22.878Z cpu39:2097995)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x4d, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T02:07:22.878Z cpu39:2097723)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x4e, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T02:08:15.577Z cpu28:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T02:14:15.577Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T02:15:42.302Z cpu32:2097243)ScsiDeviceIO: 3435: Cmd(0x45a247c64e80) 0x1a, CmdSN 0xb52 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T02:24:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T02:34:15.578Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T02:37:23.153Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x52, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T02:37:23.153Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x53, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T02:38:15.578Z cpu6:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T02:44:15.578Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T02:54:15.578Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T03:04:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T03:07:23.429Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x57, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T03:07:23.429Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x58, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T03:08:15.577Z cpu16:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T03:10:50.360Z cpu21:2097232)ScsiDeviceIO: 3435: Cmd(0x45a240faa740) 0x1a, CmdSN 0xc20 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T03:14:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T03:24:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T03:34:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T03:37:23.706Z cpu20:2097821)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x5c, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T03:37:23.706Z cpu20:2097821)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x5d, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T03:38:15.577Z cpu15:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T03:44:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T03:54:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T04:04:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T04:07:23.972Z cpu26:2097237)ScsiDeviceIO: 3435: Cmd(0x45a240e18080) 0x85, CmdSN 0x60 from world 2099626 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.

2020-03-23T04:07:23.974Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x61, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T04:07:23.974Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x62, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T04:08:15.577Z cpu2:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T04:14:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T04:24:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T04:34:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T04:37:24.243Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x66, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T04:37:24.243Z cpu20:2097994)ScsiDeviceIO: 3435: Cmd(0x45a240e47d80) 0x85, CmdSN 0x66 from world 2099626 to dev "naa.600508b1001c0b47bb91847955c738da" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.

2020-03-23T04:37:24.243Z cpu20:2097649)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x67, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T04:38:15.575Z cpu3:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T04:44:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T04:54:15.577Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T05:04:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T05:07:24.336Z cpu22:2097233)ScsiDeviceIO: 3435: Cmd(0x45a240e794c0) 0x1a, CmdSN 0xdba from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T05:07:24.510Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x6b, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:07:24.511Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x6c, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:08:15.576Z cpu1:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T05:14:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T05:24:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T05:34:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T05:37:24.777Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x70, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:37:24.777Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x71, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T05:38:15.576Z cpu4:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T05:44:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T05:54:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T06:01:00.763Z cpu30:2097241)ScsiDeviceIO: 3435: Cmd(0x459a4b210e40) 0x1a, CmdSN 0xe86 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T06:04:15.574Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T06:07:25.055Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x75, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:07:25.055Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x76, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:08:15.576Z cpu29:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T06:14:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T06:24:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T06:34:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T06:37:25.332Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x7a, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:37:25.332Z cpu20:2099626)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x7b, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T06:38:15.576Z cpu24:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T06:44:15.576Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T06:54:15.575Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T07:00:55.535Z cpu39:2097250)ScsiDeviceIO: 3435: Cmd(0x459a4b2bfbc0) 0x1a, CmdSN 0xf7c from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T07:04:15.575Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T07:07:25.610Z cpu20:2097943)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x7f, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:07:25.610Z cpu20:2097943)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x80, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:08:15.575Z cpu37:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T07:14:15.575Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T07:24:15.575Z cpu11:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T07:34:15.575Z cpu2:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T07:37:25.878Z cpu34:2097441)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x84, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:37:25.878Z cpu34:2097649)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x85, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T07:38:15.575Z cpu4:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T07:44:15.575Z cpu2:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T07:50:57.696Z cpu39:2097250)ScsiDeviceIO: 3435: Cmd(0x45a240ec4880) 0x1a, CmdSN 0x1044 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

2020-03-23T07:54:15.575Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T08:04:15.575Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

2020-03-23T08:07:26.142Z cpu34:2097186)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x89, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:07:26.142Z cpu34:2097649)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x8a, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:08:15.575Z cpu5:2097883)NMP: nmp_ResetDeviceLogThrottling:3580: Error status H:0x0 D:0x2 P:0x0 Sense Data: 0x5 0x20 0x0 from dev "mpx.vmhba32:C0:T0:L0" occurred 1 times(of 1 commands)

2020-03-23T08:10:21.521Z cpu38:2099738 opID=ee9b874d)World: 11943: VC opID c804d116 maps to vmkernel opID ee9b874d

2020-03-23T08:10:21.521Z cpu38:2099738 opID=ee9b874d)nhpsa: hpsa_SetupCmdSG:2253: Max SG entries since init: 257

2020-03-23T08:14:15.575Z cpu8:2097948)DVFilter: 5963: Checking disconnected filters for timeouts

Now i am trying to update NIC driver on new host

Reply
0 Kudos
Nawals
Expert
Expert
Jump to solution

On ESXi 225 have SCSI sense problem on disk mpx.vmhba32:C0:T0:L0  I believe This is local SD card where installed ESXi. Please check with vendor and replace the faulty part to fix it.

Host 217 vmkernel log:

Line 15: 2020-03-23T01:30:40.165Z cpu20:2097231)ScsiDeviceIO: 3435: Cmd(0x45a247c3d4c0) 0x1a, CmdSN 0xa84 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 39: 2020-03-23T02:15:42.302Z cpu32:2097243)ScsiDeviceIO: 3435: Cmd(0x45a247c64e80) 0x1a, CmdSN 0xb52 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 63: 2020-03-23T03:10:50.360Z cpu21:2097232)ScsiDeviceIO: 3435: Cmd(0x45a240faa740) 0x1a, CmdSN 0xc20 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 111: 2020-03-23T05:07:24.336Z cpu22:2097233)ScsiDeviceIO: 3435: Cmd(0x45a240e794c0) 0x1a, CmdSN 0xdba from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 135: 2020-03-23T06:01:00.763Z cpu30:2097241)ScsiDeviceIO: 3435: Cmd(0x459a4b210e40) 0x1a, CmdSN 0xe86 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 161: 2020-03-23T07:00:55.535Z cpu39:2097250)ScsiDeviceIO: 3435: Cmd(0x459a4b2bfbc0) 0x1a, CmdSN 0xf7c from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 185: 2020-03-23T07:50:57.696Z cpu39:2097250)ScsiDeviceIO: 3435: Cmd(0x45a240ec4880) 0x1a, CmdSN 0x1044 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0

Host 225 vmkernel log:

Line 15: 2020-03-23T01:30:40.165Z cpu20:2097231)ScsiDeviceIO: 3435: Cmd(0x45a247c3d4c0) 0x1a, CmdSN 0xa84 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 39: 2020-03-23T02:15:42.302Z cpu32:2097243)ScsiDeviceIO: 3435: Cmd(0x45a247c64e80) 0x1a, CmdSN 0xb52 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 63: 2020-03-23T03:10:50.360Z cpu21:2097232)ScsiDeviceIO: 3435: Cmd(0x45a240faa740) 0x1a, CmdSN 0xc20 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 111: 2020-03-23T05:07:24.336Z cpu22:2097233)ScsiDeviceIO: 3435: Cmd(0x45a240e794c0) 0x1a, CmdSN 0xdba from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 135: 2020-03-23T06:01:00.763Z cpu30:2097241)ScsiDeviceIO: 3435: Cmd(0x459a4b210e40) 0x1a, CmdSN 0xe86 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 161: 2020-03-23T07:00:55.535Z cpu39:2097250)ScsiDeviceIO: 3435: Cmd(0x459a4b2bfbc0) 0x1a, CmdSN 0xf7c from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Line 185: 2020-03-23T07:50:57.696Z cpu39:2097250)ScsiDeviceIO: 3435: Cmd(0x45a240ec4880) 0x1a, CmdSN 0x1044 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

Here is the validated SCSI sense code:

pastedImage_60.png

Please mark helpful or correct if your issue resolved.

NKS Please Mark Helpful/correct if my answer resolve your query.
Nawals
Expert
Expert
Jump to solution

Noein​ - Additionally Please see below.

2020-03-23T08:07:26.142Z cpu34:2097649)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x8a, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T04:37:24.243Z cpu20:2097994)ScsiDeviceIO: 3435: Cmd(0x45a240e47d80) 0x85, CmdSN 0x66 from world 2099626 to dev "naa.600508b1001c0b47bb91847955c738da" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.

2020-03-23T08:07:26.142Z cpu34:2097649)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x8a, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:07:26.142Z cpu34:2097649)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x8a, worldId: 0x2009aa, Cmd: 0x4d, ASC: 0x20, ASCQ: 0x0

2020-03-23T01:37:22.613Z cpu39:2097994)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x48, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

2020-03-23T08:07:26.142Z cpu34:2097186)nhpsa: hpsa_vmkScsiCmdDone:6406: Sense data: error code: 0x70, key: 0x5, info:00 00 00 00 , cmdInfo:00 00 00 00 , CmdSN: 0x89, worldId: 0x2009aa, Cmd: 0x85, ASC: 0x20, ASCQ: 0x0

you can translate it to below form.

Valid sense data : 0x5 0x20 0x0

This message is generated because your disk or device does not support 0x85 command. 0x85 is ATA PASS-THROUGH(16)

you can safely ignore it.

Understanding SCSI Check Conditions in VMkernel logs during rescan operations (1010244) | VMware KB

ASQ/ASCQ 0x20/0x0 is Invalid Command Operation Code

ASQ/ASCQ 0x24/0x0 is Invalid Field in CDB (Command Descriptor Block)

This is not necessarily a problem. Some devices do not and cannot provide the requested information

If the storage device is otherwise working correctly, the log messages can be ignored.

Please check with VMware as well as hardware vendor to upgrade ASM bundle back upgrade...

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
berndweyand
Expert
Expert
Jump to solution

do you use the same tcp/ip-stack for vmotion on all hosts ? that means default-stack or vmotion-stack?

Reply
0 Kudos
Nawals
Expert
Expert
Jump to solution

Noein​ - Can You please share both ESXi hosts connectivity details along with vSwitch, Mgmt, vMotion and network security policy screenshots.

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
Noein
Enthusiast
Enthusiast
Jump to solution

berndweyand have tested both variants same result Smiley Sad

Nawals​ i will post creenshots in half of an hour

btw i have VMware ESXi, 6.7.0, 15160138 on new host, and VMware ESXi, 6.7.0, 8169922 on old host.

And new host shows me message esx.problem.hyperthreading.unmitigated , there was not such message when there was VMware ESXi, 6.7.0, 8169922 on new host earlyer before reinstallation

So, may be patches that fix CPU vurnalebles, broke ESXi vMotion and it is nessesary instal earlyer version of ESXi on new host, or newer version of ESXi on old host?

Reply
0 Kudos
Noein
Enthusiast
Enthusiast
Jump to solution

Here is my configuration.

Am i understand right, that security policy is tab "Security" in "vSwitchN - Edit Settings" window?

Reply
0 Kudos
Nawals
Expert
Expert
Jump to solution

Make sure both ESXi with same build. this could be the one reason for failing migration. Please update the ESXi to make compliance.

NKS Please Mark Helpful/correct if my answer resolve your query.
Nawals
Expert
Expert
Jump to solution

On both vSwitch0 and vSwitch1 showing identical configuration. BTW are you using any physical switch or If yes can you please check the MTU size on Switch side.

NKS Please Mark Helpful/correct if my answer resolve your query.
Noein
Enthusiast
Enthusiast
Jump to solution

for vMotion i am using vSwitch1 on vminc1 . This ports are connected one to another directly with a cabel. Today at night i will reinstall ESXi to older version. Cannot risk with one working server, 1st will try to rollback version of new host and look what happens

Reply
0 Kudos
Nawals
Expert
Expert
Jump to solution

OK. Fine. Once you re-installed make the same configuration as old and then try to migrate the VM. If earlier working then sure will work.

NKS Please Mark Helpful/correct if my answer resolve your query.
Reply
0 Kudos
Noein
Enthusiast
Enthusiast
Jump to solution

Nawals Thank you very much!!!!!!!! I have rolled back new server from Mware ESXi, 6.7.0, 15160138 to 6.7.0, 8169922, so now i have 2 ESXi hosts with 8169922 build. Configured everything as on screenshots and everything works agein!!