So im in the process of upgrading our ESX hosts to the latest patch level for 3.0.2. The other 3 servers have gone fine, but this one is having some problems with vmotion.

It's doing the old "fail vmotion at 10% trick" which I know is nearly always network related, but I can't see where i'm messing up.

If I have my vmotion port on it's own vswitch it works, if i move the vmotion port to a shared switch with the COS & NFS ports vmotion fails. I'm using the same NIC both times, so I dont see how it can be physical switch config issues, the ports are trunked. I'm using the same VLAN ID, IP, gateway, mask etc etc..

I can vmkping between all my hosts using both shortname and FQDN. I've rebooted the server, i've tried removing the vmotion port and readding it. I've tried with multiple VMs from multiple ESX hosts..and still no go.

Example from the vmware.log of one of the guests i tried to vmotion:

Apr 22 12:31:38.018: vmx| MigrateWaitForData: waiting for data.
Apr 22 12:31:38.018: vmx| MigrateStateUpdate: Transitioning from state 7 to 8.
Apr 22 12:32:38.019: vmx| MigrateWaitForData: Waited for 60.00 seconds.
Apr 22 12:32:38.019: vmx| MigrateWaitForData: timed out. Migration has failed
Apr 22 12:32:38.019: vmx| MigrateStatusFailure: Timed out waiting for migration data.
Apr 22 12:32:38.020: vmx| MigrateSetInfo: state=11 srcIp=<0.0.0.0> dstIp=<0.0.0.0> mid=0 uuid=(null) priority=(null)
Apr 22 12:32:38.020: vmx| MigrateStateUpdate: Transitioning from state 8 to 11.
Apr 22 12:32:38.020: vmx| Migrate_ClearDoneState: cleared state. State was 11.
Apr 22 12:32:38.020: vmx| MigrateStateUpdate: Transitioning from state 11 to 0.
Apr 22 12:32:38.020: vmx| Module Migrate power on failed.
Apr 22 12:32:38.020: vmx| VMX_PowerOn: ModuleTable_PowerOn = 0
Apr 22 12:32:38.296: vmx| vmdbPipe_Streams Couldn't read: OVL_STATUS_EOF
Apr 22 12:32:38.297: vmx| VMX idle exit
Apr 22 12:32:38.310: vmx| Flushing VMX VMDB connections
Apr 22 12:32:38.400: vmx| IPC_exit: disconnecting all threads
Apr 22 12:32:38.400: vmx| VMX exit.

And from hostd.log on the source ESX server:

2008-04-22 12:32:53.401 'vm:/vmfs/volumes/45a8f531-2cd8c4fd-3541-000d60981fc6/AJ-XPBUILD2/AJ-XPBUILD2.vmx' 5417904 verbose VMotionLastStatusCb: Failed with error 536870944: Migration failed while copying data. Timeout.
2008-04-22 12:32:53.415 'ha-eventmgr' 22731696 info Event 12686 : Message on IPTV-MGMT on esx3.xxxxx.com in ha-datacenter: Migration to host <10.110.251.11> failed with error Timeout (0xbad0020)
2008-04-22 12:32:53.589 'vm:/vmfs/volumes/45a8f531-2cd8c4fd-3541-000d60981fc6/AJ-XPBUILD2/AJ-XPBUILD2.vmx' 22731696 verbose Auto-answered question Migration to host <10.110.251.11> failed with error Timeout (0xbad0020)

Any help or things to try much appreciated!