VMware Cloud Community
JavAlv
Contributor
Contributor

Storage vMotion Error

Hi,

I am having an issue when trying to do a Storage vMotion to a VM I have a ESXi 6.0 Build 3620759 with vCenter 6.0 Build 5318203 the thing is that I am able to start the SvMotion but it shows an error at 99% with the following message:

The source detected that the destination failed to resume.

I check the UUID of my storages and they are the same, also remove the VM from the inventory and add it againg and I see the same error. I don't understand 'cause I can move some machines from Datastore1 to Datastore2 with no problem but this machine and some other give me the same error. I check the vmware.log and this is what I found:

2017-11-08T01:14:22.740Z| Worker#1| I120: Migrate: Remote Log: Destination waited for 2512.62 seconds.

2017-11-08T01:14:22.749Z| vmx| I120: VMXVmdb_SetMigrationHostLogState: hostlog state transits to failure for migrate 'to' mid 1510088963335479

2017-11-08T01:14:22.901Z| vmx| I120: MigrateSetStateFinished: type=1 new state=6

2017-11-08T01:14:22.901Z| vmx| I120: MigrateSetState: Transitioning from state 3 to 6.

2017-11-08T01:14:22.901Z| vmx| A115: ConfigDB: Setting config.readOnly = "FALSE"

2017-11-08T01:14:22.901Z| vmx| I120: Migrate_SetFailureMsgList: switching to new log file.

2017-11-08T01:14:22.902Z| vmx| I120: Migrate_SetFailureMsgList: Now in new log file.

2017-11-08T01:14:22.903Z| vmx| I120: Migrate: Caching migration error message list:

2017-11-08T01:14:22.903Z| vmx| I120: [msg.migrate.fail.dst] The source detected that the destination failed to resume.

Before that I can see a really low R/W speed:

2017-11-08T00:32:31.708Z| vcpu-2| I120: PVSCSI: scsi0: switching to sync

2017-11-08T00:32:41.411Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 0 MB at 0 kB/s.

2017-11-08T00:32:51.415Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6550 kB/s over the last 10.00 seconds, copied total 64 MB at 3274 kB/s.

2017-11-08T00:33:11.418Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6551 kB/s over the last 10.00 seconds, copied total 640 MB at 16377 kB/s.

2017-11-08T00:33:21.429Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 640 MB at 13100 kB/s.

2017-11-08T00:33:31.434Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 640 MB at 10916 kB/s.

2017-11-08T00:33:41.437Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6552 kB/s over the last 10.00 seconds, copied total 704 MB at 10293 kB/s.

2017-11-08T00:33:51.448Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6546 kB/s over the last 10.01 seconds, copied total 768 MB at 9824 kB/s.

2017-11-08T00:34:01.455Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 768 MB at 8733 kB/s.

2017-11-08T00:34:11.457Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6552 kB/s over the last 10.00 seconds, copied total 832 MB at 8515 kB/s.

2017-11-08T00:36:11.511Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6546 kB/s over the last 10.01 seconds, copied total 48064 MB at 223605 kB/s.

2017-11-08T00:37:01.538Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 48832 MB at 185106 kB/s.

2017-11-08T00:37:11.546Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 48832 MB at 178493 kB/s.

2017-11-08T00:37:31.549Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 48960 MB at 167034 kB/s.

2017-11-08T00:37:41.556Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6549 kB/s over the last 10.01 seconds, copied total 49024 MB at 161857 kB/s.

2017-11-08T00:38:01.570Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6547 kB/s over the last 10.01 seconds, copied total 49216 MB at 152641 kB/s.

2017-11-08T00:38:11.573Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6551 kB/s over the last 10.00 seconds, copied total 49280 MB at 148345 kB/s.

2017-11-08T00:38:21.582Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 49280 MB at 144105 kB/s.

2017-11-08T00:39:01.600Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 50048 MB at 131341 kB/s.

2017-11-08T00:39:11.606Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6549 kB/s over the last 10.01 seconds, copied total 50112 MB at 128221 kB/s.

2017-11-08T00:39:21.614Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6548 kB/s over the last 10.01 seconds, copied total 50176 MB at 125252 kB/s.

2017-11-08T00:39:41.623Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6548 kB/s over the last 10.01 seconds, copied total 50496 MB at 120189 kB/s.

2017-11-08T00:40:21.648Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 50880 MB at 110795 kB/s.

2017-11-08T00:40:41.658Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6551 kB/s over the last 10.00 seconds, copied total 51136 MB at 106807 kB/s.

2017-11-08T00:40:51.661Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 51136 MB at 104672 kB/s.

2017-11-08T00:41:01.663Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6552 kB/s over the last 10.00 seconds, copied total 51200 MB at 102749 kB/s.

2017-11-08T00:41:11.670Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6549 kB/s over the last 10.01 seconds, copied total 51264 MB at 100898 kB/s.

2017-11-08T00:41:21.673Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6551 kB/s over the last 10.00 seconds, copied total 51328 MB at 99118 kB/s.

2017-11-08T00:41:31.673Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6553 kB/s over the last 10.00 seconds, copied total 51392 MB at 97405 kB/s.

2017-11-08T00:41:41.685Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6545 kB/s over the last 10.01 seconds, copied total 51456 MB at 95752 kB/s.

2017-11-08T00:41:51.694Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6547 kB/s over the last 10.01 seconds, copied total 51520 MB at 94158 kB/s.

2017-11-08T00:42:01.697Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 51520 MB at 92507 kB/s.

2017-11-08T00:42:31.710Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 51904 MB at 88537 kB/s.

2017-11-08T00:42:41.719Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6547 kB/s over the last 10.01 seconds, copied total 51968 MB at 87192 kB/s.

2017-11-08T00:42:51.720Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 51968 MB at 85787 kB/s.

2017-11-08T00:43:01.723Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 51968 MB at 84425 kB/s.

2017-11-08T00:43:11.724Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6552 kB/s over the last 10.00 seconds, copied total 52032 MB at 83209 kB/s.

2017-11-08T00:43:21.731Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52032 MB at 81929 kB/s.

2017-11-08T00:43:31.734Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52032 MB at 80687 kB/s.

2017-11-08T00:43:41.737Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52032 MB at 79483 kB/s.

2017-11-08T00:43:51.738Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52032 MB at 78315 kB/s.

2017-11-08T00:44:01.747Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52032 MB at 77179 kB/s.

2017-11-08T00:44:11.753Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52032 MB at 76077 kB/s.

2017-11-08T00:44:21.757Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6550 kB/s over the last 10.00 seconds, copied total 52096 MB at 75098 kB/s.

2017-11-08T00:44:31.766Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52096 MB at 74054 kB/s.

2017-11-08T00:44:41.766Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52096 MB at 73040 kB/s.

2017-11-08T00:44:51.768Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52096 MB at 72054 kB/s.

2017-11-08T00:45:01.768Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52096 MB at 71093 kB/s.

2017-11-08T00:45:11.776Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6548 kB/s over the last 10.01 seconds, copied total 52160 MB at 70244 kB/s.

2017-11-08T00:45:21.788Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6545 kB/s over the last 10.01 seconds, copied total 52224 MB at 69416 kB/s.

2017-11-08T00:45:31.790Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52224 MB at 68526 kB/s.

2017-11-08T00:45:41.797Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52224 MB at 67659 kB/s.

2017-11-08T00:45:51.797Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52224 MB at 66813 kB/s.

2017-11-08T00:46:01.807Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52224 MB at 65988 kB/s.

2017-11-08T00:46:11.807Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6553 kB/s over the last 10.00 seconds, copied total 52288 MB at 65264 kB/s.

2017-11-08T00:46:21.813Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52288 MB at 64477 kB/s.

2017-11-08T00:46:31.817Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52288 MB at 63710 kB/s.

2017-11-08T00:46:41.819Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52288 MB at 62960 kB/s.

2017-11-08T00:46:51.821Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52288 MB at 62228 kB/s.

2017-11-08T00:47:01.823Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52288 MB at 61513 kB/s.

2017-11-08T00:47:11.829Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6549 kB/s over the last 10.01 seconds, copied total 52352 MB at 60889 kB/s.

2017-11-08T00:47:21.838Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52352 MB at 60204 kB/s.

2017-11-08T00:47:31.838Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52352 MB at 59536 kB/s.

2017-11-08T00:47:41.846Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52352 MB at 58881 kB/s.

2017-11-08T00:47:51.855Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52352 MB at 58241 kB/s.

2017-11-08T00:48:01.857Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52352 MB at 57615 kB/s.

2017-11-08T00:48:11.859Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6551 kB/s over the last 10.00 seconds, copied total 52416 MB at 57072 kB/s.

2017-11-08T00:48:21.868Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52416 MB at 56471 kB/s.

2017-11-08T00:48:31.877Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6548 kB/s over the last 10.01 seconds, copied total 52480 MB at 55951 kB/s.

2017-11-08T00:48:41.882Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 55374 kB/s.

2017-11-08T00:48:51.892Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 54808 kB/s.

2017-11-08T00:49:01.896Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52480 MB at 54255 kB/s.

2017-11-08T00:49:11.907Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 53712 kB/s.

2017-11-08T00:49:21.914Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 53180 kB/s.

2017-11-08T00:49:31.922Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 52658 kB/s.

2017-11-08T00:49:41.931Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 52147 kB/s.

2017-11-08T00:49:51.941Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 51645 kB/s.

2017-11-08T00:50:01.952Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 51153 kB/s.

2017-11-08T00:50:11.963Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 50670 kB/s.

2017-11-08T00:50:21.964Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52480 MB at 50197 kB/s.

2017-11-08T00:50:31.972Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 49732 kB/s.

2017-11-08T00:50:41.972Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52480 MB at 49276 kB/s.

2017-11-08T00:50:47.500Z| vcpu-1| I120: PVSCSI: scsi0:0: aborting cmd 0xea

2017-11-08T00:50:51.978Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 48828 kB/s.

2017-11-08T00:51:01.986Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52480 MB at 48388 kB/s.

2017-11-08T00:51:11.988Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6551 kB/s over the last 10.00 seconds, copied total 52544 MB at 48015 kB/s.

2017-11-08T00:51:21.999Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52544 MB at 47589 kB/s.

2017-11-08T00:51:32.000Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52544 MB at 47172 kB/s.

2017-11-08T00:51:42.007Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52544 MB at 46762 kB/s.

2017-11-08T00:51:52.013Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52544 MB at 46359 kB/s.

2017-11-08T00:52:02.013Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52544 MB at 45963 kB/s.

2017-11-08T00:52:12.023Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52544 MB at 45573 kB/s.

2017-11-08T00:52:22.032Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6547 kB/s over the last 10.01 seconds, copied total 52608 MB at 45245 kB/s.

2017-11-08T00:52:32.040Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52608 MB at 44868 kB/s.

2017-11-08T00:52:42.049Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52608 MB at 44497 kB/s.

2017-11-08T00:52:52.059Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52608 MB at 44132 kB/s.

2017-11-08T00:53:02.067Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52608 MB at 43773 kB/s.

2017-11-08T00:53:12.073Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52608 MB at 43420 kB/s.

2017-11-08T00:53:22.078Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52608 MB at 43073 kB/s.

2017-11-08T00:53:32.087Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 6547 kB/s over the last 10.01 seconds, copied total 52672 MB at 42783 kB/s.

2017-11-08T00:53:42.090Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 52672 MB at 42446 kB/s.

2017-11-08T00:53:52.099Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 52672 MB at 42114 kB/s.

2017-11-08T01:14:02.783Z| vmx| W110: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 58816 MB at 24174 kB/s.

Can that cause this issue to happen?

I'll apreciate any help.

Gracias,

Reply
0 Kudos
2 Replies
mhampto
VMware Employee
VMware Employee

Does the VM have snapshots? Does this happen with multiple VM's? Can you manually copy data from the host between the storage locations? This initially looks like a storage issue, it may be worth while running the copy and vMotion tests to open esxtop (VMware Knowledge Base )

Reply
0 Kudos
vXav
Expert
Expert

Did you ever find what was going on?

I'm getting the same thing on one VM with very similar transfer rates interestingly.

Reply
0 Kudos