Unable to do sVmotion for all VMs provisioned with virtual RDMs in vCenter 5.5. Error message: A fatal internal error occurred. See the virtual machine's log for more details.

[SDquality]

vmware.log

=========

2014-07-10T12:22:19.043Z| vcpu-0| I120: HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmdk'

2014-07-10T12:22:19.043Z| vcpu-0| I120: DDB: "longContentID" = "620524f4ef571fba22724899e61b106e" (was "1e69ec0479818e7584abfdfc1cc342eb")

2014-07-10T12:22:19.066Z| vcpu-0| I120: DISKLIB-CHAIN : DiskChainUpdateContentID: old=0x1cc342eb, new=0xe61b106e (620524f4ef571fba22724899e61b106e)

2014-07-10T12:23:22.050Z| vmx| I120: SVMotion: scsi0:0: Disk copy completed for total 32768 MB at 466072 kB/s.

2014-07-10T12:23:22.066Z| Worker#1| I120: Disk/File copy started for /vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_1.vmdk.

2014-07-10T12:23:26.189Z| vmx| I120: SVMotion: scsi0:1: Disk copy completed for total 32768 MB at 8137272 kB/s.

2014-07-10T12:23:26.190Z| Worker#1| I120: Disk/File copy started for /vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02-ctk.vmdk.

2014-07-10T12:23:26.223Z| vmx| I120: SVMotion: File copy completed for /vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02-ctk.vmdk

2014-07-10T12:23:26.225Z| Worker#1| I120: Disk/File copy started for /vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_1-ctk.vmdk.

2014-07-10T12:23:26.235Z| vmx| I120: SVMotion: File copy completed for /vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_1-ctk.vmdk

2014-07-10T12:23:26.235Z| Worker#1| W110: SVMotionMirroredModeAsyncCopyIoctl: Cannot copy source (/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk) to destination (/vmfs/volumes/53b6a7b7-bffe4b0b

-27ff-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk): One of the parameters supplied is invalid

2014-07-10T12:23:26.235Z| Worker#1| W110: SVMotionAsyncCopyInit: Failed to issue copy start from source (/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk) to destination (/vmfs/volumes/53b6a7b7-b

ffe4b0b-27ff-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk).

2014-07-10T12:23:26.235Z| Worker#1| W110: SVMotionMirroredModeAsyncCopyAllFiles:7841. Failed to issue async file copy start ioctl for file /vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk.

2014-07-10T12:23:26.235Z| Worker#1| W110: SVMotionMirroredModeThreadDiskCopy: Failed to copy files via mirror module.

2014-07-10T12:23:26.235Z| Worker#1| W110: SVMotionMirroredMode: disk copy failed. Canceling Storage vMotion.

2014-07-10T12:23:26.235Z| vmx| I120: SVMotionMirroredMode: Worker thread exited.

2014-07-10T12:23:26.236Z| vmx| I120: VMXVmdb_SetMigrationHostLogState: hostlog state transits to failure for migrate 'to' mid 1404994925548955

2014-07-10T12:23:26.255Z| vmx| I120: MigrateSetStateFinished: type=1 new state=5

2014-07-10T12:23:26.255Z| vmx| I120: MigrateSetState: Transitioning from state 2 to 5.

2014-07-10T12:23:26.255Z| vmx| I120: Migrate_SetFailureMsgList: switching to new log file.

2014-07-10T12:23:26.269Z| vmx| I120: Migrate_SetFailureMsgList: Now in new log file.

2014-07-10T12:23:26.284Z| vmx| I120: [msg.svmotion.fail.internal] A fatal internal error occurred. See the virtual machine's log for more details.

2014-07-10T12:23:26.285Z| vmx| I120: [msg.svmotion.disk.copyphase.failed] Failed to copy one or more disks.

2014-07-10T12:23:26.285Z| vmx| I120: Migrate: cleaning up migration state.

2014-07-10T12:23:26.285Z| vmx| I120: SVMotion: Enter Phase 11

2014-07-10T12:23:26.285Z| vmx| I120: SVMotion_Cleanup: Scheduling cleanup thread.

2014-07-10T12:23:26.285Z| Worker#0| I120: SVMotionCleanupThread: Waiting for SVMotion Bitmap thread to complete.

2014-07-10T12:23:26.285Z| vmx| I120: MigrateSetState: Transitioning from state 5 to 0.

2014-07-10T12:23:26.285Z| vcpu-0| I120: Destroying virtual dev for scsi0:0 vscsi=12578

2014-07-10T12:23:26.285Z| vcpu-0| I120: VMMon_VSCSIStopVports: No such target on adapter

2014-07-10T12:23:26.285Z| vcpu-0| I120: Destroying virtual dev for scsi0:1 vscsi=12579

2014-07-10T12:23:26.285Z| vcpu-0| I120: VMMon_VSCSIStopVports: No such target on adapter

2014-07-10T12:23:26.285Z| vcpu-0| I120: Destroying virtual dev for scsi0:2 vscsi=12580

2014-07-10T12:23:26.285Z| vcpu-0| I120: VMMon_VSCSIStopVports: No such target on adapter

2014-07-10T12:23:26.297Z| Worker#0| I120: SVMotionCleanupThread: Waiting for SVMotion thread to complete.

2014-07-10T12:23:26.297Z| Worker#0| I120: SVMotionCleanupThread: Waiting for final stun/unstun to finish

2014-07-10T12:23:26.298Z| vmx| I120: MigrateESXVmkEventHandler: migration not in progress, not handling message 3.

2014-07-10T12:23:26.298Z| vcpu-0| I120: Closing all the disks of the VM.

2014-07-10T12:23:26.298Z| vcpu-0| I120: Closing disk scsi0:2

2014-07-10T12:23:26.302Z| vcpu-0| I120: DISKLIB-CBT : Shutting down change tracking for untracked fid 60125232.

2014-07-10T12:23:26.302Z| vcpu-0| I120: DISKLIB-CBT : Successfully disconnected CBT node.

2014-07-10T12:23:26.329Z| vcpu-0| I120: DISKLIB-VMFS : "/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-rdm.vmdk" : closed.

2014-07-10T12:23:26.329Z| vcpu-0| I120: Closing disk scsi0:0

2014-07-10T12:23:26.330Z| vcpu-0| I120: DISKLIB-CBT : Shutting down change tracking for untracked fid 78999588.

2014-07-10T12:23:26.330Z| vcpu-0| I120: DISKLIB-CBT : Successfully disconnected CBT node.

2014-07-10T12:23:26.334Z| vcpu-0| I120: DISKLIB-VMFS : "/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02-flat.vmdk" : closed.

2014-07-10T12:23:26.334Z| vcpu-0| I120: Closing disk scsi0:1

2014-07-10T12:23:26.335Z| vcpu-0| I120: DISKLIB-CBT : Shutting down change tracking for untracked fid 58224682.

hostd.log

=========

2014-07-10T12:23:26.371Z [FFF47B70 verbose 'Vmsvc.vm:/vmfs/volumes/53b6a7b7-bffe4b0b-27ff-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] Closed VM handle.

2014-07-10T12:23:26.411Z [3ED80B70 verbose 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] Time to gather config: 38 (msecs)

2014-07-10T12:23:26.412Z [FFF47B70 verbose 'Hbrsvc'] Replicator: ReconfigListener triggered for config VM 23

2014-07-10T12:23:26.413Z [3ED80B70 verbose 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] VMotionLastStatusCb:: hostlog state changed from emigrating to failure

2014-07-10T12:23:26.414Z [FFF47B70 verbose 'Hbrsvc'] Replicator: VmReconfig ignoring VM 23 not configured for replication

2014-07-10T12:23:26.416Z [3EB80B70 verbose 'Default' opID=SWI-6f342d8b user=vpxuser] AdapterServer: target='vim.ResourcePool:ha-root-pool', method='GetConfig'

2014-07-10T12:23:26.417Z [3E180B70 verbose 'Default' opID=SWI-6f342d8b user=vpxuser] AdapterServer: target='vim.ResourcePool:ha-root-pool', method='GetName'

2014-07-10T12:23:26.425Z [3E180B70 verbose 'Default' opID=31baa34d-6d-ed-37 user=vpxuser] AdapterServer: target='vmodl.query.PropertyCollector.Filter:session[066ff19e-189c-c5b9-1886-acf7ebd0d119]52b15255-e95f-8ddf-d4f3-ac9f4e97fabc', met

hod='destroy'

2014-07-10T12:23:26.576Z [3BCD2B70 info 'vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] Disconnect check in progress.

2014-07-10T12:23:26.576Z [3EB80B70 verbose 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] VMotionLastStatusCb: Failed with error 340983807: A fatal internal error occurred. See the virtual

machine's log for more details.

--> Failed to copy one or more disks.

-->

2014-07-10T12:23:26.576Z [3EB80B70 verbose 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] VMotionResolveCheck: Operation in progress

2014-07-10T12:23:26.576Z [3EB80B70 verbose 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] VMotionStatusCb: Completed

2014-07-10T12:23:26.576Z [3EB80B70 verbose 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] VMotionResolveCheck: Firing ResolveCb

2014-07-10T12:23:26.576Z [3EB80B70 info 'Vcsvc.VMotionSrc (1404994925548955)'] ResolveCb: VMX reports needsUnregister = false for migrateType MIGRATE_TYPE_FSR

2014-07-10T12:23:26.576Z [3EB80B70 info 'Vcsvc.VMotionSrc (1404994925548955)'] ResolveCb: Failed with fault: (vmodl.fault.SystemError) {

--> dynamicType = <unset>,

--> faultCause = (vmodl.MethodFault) null,

--> reason = "A fatal internal error occurred. See the virtual machine's log for more details.

--> Failed to copy one or more disks.

--> ",

--> msg = "",

--> }

2014-07-10T12:23:26.576Z [3EB80B70 verbose 'Vcsvc.VMotionSrc (1404994925548955)'] Migration changed state from MIGRATING to DONE

2014-07-10T12:23:26.576Z [3EB80B70 verbose 'Vcsvc.VMotionSrc (1404994925548955)'] Finish called

2014-07-10T12:23:26.576Z [3EB80B70 info 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] State Transition (VM_STATE_EMIGRATING -> VM_STATE_ON)

2014-07-10T12:23:26.577Z [3AF40B70 verbose 'Hostsvc'] Received state change for VM '23'

2014-07-10T12:23:26.577Z [3AF40B70 info 'Guestsvc.GuestFileTransferImpl'] Entered VmPowerStateListener

2014-07-10T12:23:26.577Z [3AF40B70 info 'Guestsvc.GuestFileTransferImpl'] VmPowerStateListener succeeded

2014-07-10T12:23:26.577Z [3EB80B70 info 'Hostsvc'] Decremented SIOC Injector Flag2

2014-07-10T12:23:26.577Z [3AF40B70 info 'Hbrsvc'] Replicator: powerstate change VM: 23 Old: 1 New: 1

2014-07-10T12:23:26.577Z [3EB80B70 info 'Vmsvc.vm:/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02.vmx'] VMotion cleanup completed

2014-07-10T12:23:26.577Z [3EB80B70 verbose 'Vcsvc.VMotionSrc (1404994925548955)'] JrnlEnd: Wrote active=0

2014-07-10T12:23:26.577Z [3EB80B70 verbose 'Vcsvc.VMotionSrc (1404994925548955)'] JrnlEnd: Finished journal

2014-07-10T12:23:26.577Z [3EB80B70 verbose 'Vcsvc.VMotionSrc (1404994925548955)'] [JrnlFilePersistenceProvider::DeleteJournal] Deleting /var/lib/vmware/hostd/journal/1404994928.83.

2014-07-10T12:23:26.577Z [3EB80B70 verbose 'Vcsvc.VMotionSrc (1404994925548955)'] JrnlDelete: Deleted journal

2014-07-10T12:23:26.578Z [3AF40B70 verbose 'Hbrsvc'] Replicator: Remove group no matching entry for VM (id=23)

2014-07-10T12:23:26.586Z [3D092B70 verbose 'Default' opID=31baa34d-6d-6b user=vpxuser] AdapterServer: target='vmodl.query.PropertyCollector.Filter:session[066ff19e-189c-c5b9-1886-acf7ebd0d119]5216d17e-4039-39e2-8d05-228c1c2a51c9', method

='destroy'

2014-07-10T12:23:26.597Z [3AF40B70 verbose 'Default' opID=31baa34d-6d-6c user=vpxuser] AdapterServer: target='vim.host.VMotionManager:ha-vmotionmgr', method='completeSource'

2014-07-10T12:23:26.597Z [3AF40B70 info 'Vcsvc.VMotion' opID=31baa34d-6d-6c user=vpxuser] CompleteSource [1404994925548955]

2014-07-10T12:23:26.600Z [3AF40B70 verbose 'Default' opID=31baa34d-6d-d8 user=vpxuser] AdapterServer: target='vim.host.VMotionManager:ha-vmotionmgr', method='completeDestination'

Please help!!

[virtualnobody]

You have already checked this?

VMware KB: Migrating virtual machines with Raw Device Mappings (RDMs)

This seems to e your first error:

2014-07-10T12:23:26.235Z| Worker#1| W110: SVMotionMirroredModeAsyncCopyIoctl: Cannot copy source (/vmfs/volumes/51fbb93b-fb09d917-9e5c-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk) to destination (/vmfs/volumes/53b6a7b7-bffe4b0b

-27ff-90e2ba474db4/NL-DC1-VM-PV02/NL-DC1-VM-PV02_2-ctk.vmdk): One of the parameters supplied is invalid

Check "Additional Information" in the given KB.

[SDquality]

Thank you for the reply......Is there any limitation for moving ctk.vmdk files from one datastore to other datastore.

[zsoltesz]

Did you find the reason of this svmotion errorr? I have the same.