VMotion - 90% Unknown failure migrating from another host

[cgnew]

Hei

Running VC 201/ESX 301 - trying VMotion.

When migrating evrything looks fine until 90% and reports unknown failure...

Stopping VM and then migrate - it works...

Anyone any idea ?

cg-

[MattG]

What server, HBA, and SAN hardware are you using?

-MattG

[zemotard]

I already have this problem.

I have solve it changing the scsi configuration, from bus logic to lsi logic ...

[cgnew]

HW:

2 x Dell 2950/4GB RAM

Intel 82572

SAN: Promise VTrack 500i - iSCSI

cg-

[cgnew]

scsi config is already LSI - ?

Config of scsi bus sharing ? none/virtual/physical

Which one do I choose ?

cg-

[zemotard]

Try to change your scsi configuration and restart your vm, then try a vmotion ...

For information I have 3 PE 2950 with 8gb of memory.

Just one vm had this problem.

Regarsd

[cgnew]

Hmmm - After config back to LSI - sometimes it works, sometimes it doesn't.

Strange...

Thanks for your attention. Closing subject.

regards cg-

[zemotard]

If my answer is usefull for you, please let me some points by clicking on correct or usefull buttom

[cgnew]

Sorry - there's no 'usefull bottom' ?

May be I did not mark as a question ?

cg-

[jeffwilkinson]

Do you have the HP VMM Agent running in your service console?

The vmmagent v4.1.0-1214 (and maybe other versions..) would lock the vm's swap file sometimes and if it happened during a VMotion operation, it would fail at 90%. It would happen randomly about 1 in 5 times.

[cgnew]

No - no HP VMM Agent running...

So what do I do to solve this ?

Reinstall VMware ? Some problems with latest patches ?

Any suggestions ?

[msmenne17]

What verison of ESX/VC are you running? I had this problem before, but I don't remember the exact fix.

If I remember correctly, it was an issue with one host. I ended up rebuilding that host onto 3.0.1 and upgrading the rest to 3.01. I think that ended up fixing it.

[jmcdonald1]

When vMotion Fails at 90%, this means that it failed to resume the VM on the destination system. To determine why this is happening, attempt a vMotion and let it fail, and after this happens go and look at the last 20 lines or so of the most recent vmware.log for this VM.

If you cannot see the error post the log we can take a look through it.

[sava]

Same problem I have,

Everything is latest version. No problem with dns settings.

No matter I do result is same.

during vmotion 90% stops then I see the "operation timed out"error in virtual center.

Vmotion result sometimes successfull although gives error.

Sometimes vm becomes orphaned.And powered off.

Sometimes its locks the vm and without killing the process(kill -9 pid no)

no way to release vm and re register it via SAN LUN browsing.

Sometimes I see following error in comatibility window

"Migration from esx.xx: No guest OS heartbeats are being received. Either the guest OS is not respondig or Vwmare tools is not configured properly"[/b]

/**********************vmware.log*************/

Jun 11 22:19:55.824: vcpu-0| TOOLS setting the tools version to '7201'

Jun 11 22:19:55.829: vcpu-0| TOOLS unified loop capability requested by 'toolbox'; now sending options via TCLO

Jun 11 22:19:58.395: vcpu-0| HBACommon: First write on scsi0:0.fileName/vmfs/volumes/45909d3b-b3bd6afe-ab59-001a4bdc5b5e/W2K3ADDC/W2K3ADDC.vmdk

Jun 11 22:19:58.401: vcpu-0| DISKLIB-CHAIN : UpdateContentID: old = 0xdf836b4e, new = 0x487302e

Jun 11 22:22:09.620: vmx| VMXVmdbCbVmVmxMigrate: Got SET callback for /vm/#d63d9d4a0d2e4561/vmx/migrateState/cmd/##1_b1/op/=to

Jun 11 22:22:09.620: vmx| VmxMigrateGetParam: srcIp=0xc8c8c8cb dstIp=0xc8c8c8ca mid=f7d68edd uuid=34333534-3632-435a-4a37-313330384d31 priority=low

Jun 11 22:22:09.655: vmx| MigrateSetInfo: state=1 srcIp=<200.200.200.203> dstIp=<200.200.200.202> mid=4158033629 uuid=34333534-3632-435a-4a37-313330384d31 priority=low

Jun 11 22:22:09.655: vmx| MigrateStateUpdate: Transitioning from state 0 to 1.

Jun 11 22:22:11.163: vmx| VMXVmdbCbVmVmxMigrate: Got SET callback for /vm/#d63d9d4a0d2e4561/vmx/migrateState/cmd/##1_b2/op/=start

Jun 11 22:22:11.164: vmx| VmxMigrateGetStartParam: mid=f7d68edd dstwid=1227

Jun 11 22:22:11.164: vmx| MigrateStateUpdate: Transitioning from state 1 to 2.

Jun 11 22:22:14.670: vcpu-0| MigrateStateUpdate: Transitioning from state 2 to 3.

Jun 11 22:22:14.671: vcpu-0| Sync monModules(1).

Jun 11 22:22:14.671: vcpu-0| Done Sync monModules(1).

Jun 11 22:22:14.673: vmx| DISKLIB-VMFS : "/vmfs/volumes/45909d3b-b3bd6afe-ab59-001a4bdc5b5e/W2K3ADDC/W2K3ADDC-flat.vmdk" : closed.

Jun 11 22:22:14.674: vcpu-0| Cpt monModules(2).

Jun 11 22:22:14.674: vcpu-0| Done Cpt monModules(2).

Jun 11 22:22:14.674: vmx| MigrateOpen: Migrating to <200.200.200.202> with migration id 4158033629

Jun 11 22:22:14.675: vmx| Checkpointed in VMware ESX Server, 3.0.1 build-42368, build-42368, Linux Host

Jun 11 22:22:14.680: vmx| DISKLIB-VMFS : "/vmfs/volumes/45909d3b-b3bd6afe-ab59-001a4bdc5b5e/W2K3ADDC/W2K3ADDC-flat.vmdk" : open successful (21) size = 10737418240, hd = -1. Type 3

Jun 11 22:22:14.681: vmx| DISKLIB-VMFS : "/vmfs/volumes/45909d3b-b3bd6afe-ab59-001a4bdc5b5e/W2K3ADDC/W2K3ADDC-flat.vmdk" : closed.

Jun 11 22:22:14.700: vmx| GuestMsg: Channel 0, Cannot unpost because the previous post is already completed

Jun 11 22:22:14.700: vmx| GuestRpc: Channel 0 reinitialized.

Jun 11 22:22:14.796: vmx| MigrateCloseSwap: Closing swap file.

Jun 11 22:22:14.797: vmx| Migrate: VM successfully stunned, waiting for the go/no go message.

Jun 11 22:22:14.797: vmx| MigrateStateUpdate: Transitioning from state 3 to 4.

Jun 11 22:22:15.145: vmx| MigrateSetInfo: state=5 srcIp=<0.0.0.0> dstIp=<0.0.0.0> mid=0 uuid=(null) priority=(null)

Jun 11 22:22:15.145: vmx| MigrateStateUpdate: Transitioning from state 4 to 5.

Jun 11 22:22:15.145: vmx| Migrate: Powering off

Jun 11 22:22:15.145: vmx| Stopping VCPU threads...

Jun 11 22:22:15.146: vcpu-0| VMMon_WaitForExit: vcpu-0: worldID=1145

Jun 11 22:22:15.255: mks| Async MKS thread is exiting

Jun 11 22:22:15.255: vmx| DnD rpc already set to 0

Jun 11 22:22:15.255: vmx| TOOLS received request in VMX to set option 'enableDnD' -> '0'

Jun 11 22:22:15.258: vmx| MKS local poweroff

Jun 11 22:22:15.258: vmx| Lock before MKS lock created. Early poweroff?

Jun 11 22:22:15.258: vmx| Unlock before MKS lock created. Early poweroff?

Jun 11 22:22:16.237: vmx| VMXVmdbMigrateDoneCtxCb: Waiting for 'poweredOff'. Received exec state update 'poweredOff'

Jun 11 22:22:16.237: vmx| Migrate_ClearDoneState: cleared state. State was 5.

Jun 11 22:22:16.237: vmx| MigrateStateUpdate: Transitioning from state 5 to 0.

Jun 11 22:22:16.522: vmx| SOCKET 1 client closed connection

Jun 11 22:22:16.522: vmx| vmdbPipe_Streams Couldn't read: OVL_STATUS_EOF

Jun 11 22:22:16.523: vmx| VMX idle exit

Jun 11 22:22:16.529: vmx| Flushing VMX VMDB connections

Jun 11 22:22:16.532: vmx| IPC_exit: disconnecting all threads

Jun 11 22:22:16.532: vmx| VMX exit.

Jun 11 22:22:16.532: vmx| AIOMGR-S : stat o=1 r=0 w=0 i=2 br=0 bw=0

/**********************vmware.log*************/