Folks,

I've searched through out this forum and haven't found an answer that works in the many posts out there, I figure I'd ask it again.

Here's the background:
We have 12 Identical Dell M600 Blades in 2 chassis with 16GB Ram, 2 x Xeon E5430, they are all connected to an Equallogic PS 5000XV iSCSI SAN on a separate iSCSI network (vswitch1) with 2 dedicated Nics and dedicated switch, dedicated iscsi svc console, dedicated VMkernel port for iscsi access. The Access net (vswitch0) contains the VM port groups for our different networks as well as a svc console and vmkernel port for VMotion with 2 separate NICS as well.

We are Running ESX 3.5 U3 and VCenter 2.5 U3 on Win2k3 R2

Vmotion works between all the servers, Storage Vmotion works for most of the machines, HA is working, and set to 2 host failurs with no vm monitoring, DRS is set to manual for now as I have some machines on local stores as I complete my LUN rebuilds, there are no rules set for DRS and VMware EVC is enabled for Intel hosts. However, I'm just gonna describe the one machine that needs to do svmotion below.

Here's the problem:
I am trying to Svmotion via svmotion.pl --interactive, a Windows 2000 Machine with one virtual disk and one virtual RDM. I am aware of the requirements for the RDM and the settings required for svmotion, independent is unselected for the RDM, and I have also svmotioned several machines both linux and win2k3 with the same configuration without issue. In the interactive session I choose to individually place disks, and I choose only the VM's virtual disk to be moved, basically as I've seen it will move the vm's vdisk and then copy over the RDM pointer.

The CPU utilization of this machine is around 25% average. but I try to run migrations during the lowest times. and The Host it self shows only about 5.5GB of the 16GB RAM used. so I think we are good on RAM. the volume/datastore I'm migrating from has 485GB free, and the volume/datastore I'm migrating to has 145GB free. the VM virtual disk is only about 33GB.

I run the svmotion script from the windows version of the RCLI. and when begin the process I get the following error at around 2% progress:

"Received an error from the server: A general system error occurred: Failed waiting for data. Error bad0007. Bad parameter."

After searching around I found the following fixes from the release notes of U2

* Migrate.PageInTimeoutResetOnProgress: Set the value to 1.
* Migrate.PageInProgress: Set the value to 30, if you get an error even after configuring the Migrate.PageInTimeoutResetOnProgress variable.

I made those changes and I still get the same error.

When I dig in to the log, I see these entries in the vmkwarning log:

Feb 24 00:17:32 iq-virt-c2-b6 vmkernel: 82:04:13:56.794 cpu4:1394)WARNING: Heap: 1397: Heap migHeap0 already at its maximumSize. Cannot expand.
Feb 24 00:17:32 iq-virt-c2-b6 vmkernel: 82:04:13:56.794 cpu4:1394)WARNING: Heap: 1522: Heap_Align(migHeap0, 1030120338/1030120338 bytes, 4 align) failed. caller: 0x988f61
Feb 24 00:17:32 iq-virt-c2-b6 vmkernel: 82:04:13:56.794 cpu4:1394)WARNING: Migrate: 1243: 1235452646235015: Failed: Out of memory (0xbad0014) @0x98da8b
Feb 24 00:17:32 iq-virt-c2-b6 vmkernel: 82:04:13:56.794 cpu2:1395)WARNING: MigrateNet: 309: 1235452646235015: 5-0xa023818:Sent only 0 of 4096 bytes of message data: Broken pipe
Feb 24 00:17:32 iq-virt-c2-b6 vmkernel: 82:04:13:56.794 cpu6:1396)WARNING: Migrate: 1243: 1235452646235015: Failed: Migration protocol error (0xbad003e) @0x98da8b
Feb 24 00:17:32 iq-virt-c2-b6 vmkernel: 82:04:13:56.794 cpu2:1395)WARNING: Migrate: 6776: 1235452646235015: Couldn't send data for 56486: Broken pipe

At this point I'm stuck.. Could it be the windows RCLI? the vcenter server? or the service console not having enough RAM? We've already increased all our service consoles to 512MB...

Any help would be greatly appreciated..

Thanks in advance.

Alvin