VMware Cloud Community
chrisgatguis
Contributor
Contributor

Converter 6.2 standalone failure Error: 37409 (type: 1, code: 2338

Hi all

I'm failing terribly trying to migrate a handful of VM's physical and vm's on to my vSphere 5.5 environment.

Predominantly I'm getting the error below,   If i repeat the same job (same source server - which is Windows VM, running on Esxi 4.1 host) it always fails at the same percentage each time (19% in this instance)

FAILED: An error occurred during the conversion: 'BlockLevelVolumeCloneMgr::CloneVolume: Detected a write error during the cloning of volume \WindowsBitmapDriverVolumeId=[4D-61-F8-AC-00-00-50-06-00-00-00-00]. Error: 37409 (type: 1, code:
2338)'

I've tried every conceivable change in the job config but alas I've never been able to get it complete.

I've tried:

  • standard block based with all the defaults
  • forcing file based by shrinking the disk size to a couple MB less (this VM only has an OS disk - nothing special)
  • tried slightly increasing the destination disk size, and changing to 'thin' provisioning - i got this from the release notes of v6.2 as a known issue around NFC
  • changing destination type, tried: Vcenter on vmfs datastore, also Workstation / Fusion types going to a share.  In all intances files are created but will not complete - and subsequently are removed when the job fails
  • tried running the converter standalone on the actual source server
  • Stopping SQL services on the source (stop successfully)
  • Tried running the standalone (server, worker) from three different servers now, one of which is a physical, one was the vcenter server VM, and the third just a different VM.
  • Tried using previous version of converter (v 5.0 - as its the only other installer i could find).
  • Running checkdisk on the source (didn't find anything)
  • Different source VM's of differing sizes, ranging from 100GB - 300GB
  • sync settings on / off

The error doesn't feel transient or poor network related in nature - because it always fails at the same percentage each time.

Is there anything that i can put in the config files such as timeout threshold or retry count? 

Also - is there a way of identifying which block / file it got to? If it is simply source related it may point me in the right direction (although it would need to be the same thing on multiple sources so i'm not certain it would).

thanks

Chris

Tags (2)
0 Kudos
1 Reply
chrisgatguis
Contributor
Contributor

Have also tried to limit the transfer speed (this is over an MPLS connection with 100 mbps circuit(s) between the two)... no change.

This is the agent error log from around the error:

CloneTask updates, state: 1, percentage: 19, xfer rate (Bps): 7227896

2020-06-16T10:27:16.472+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] SingleVolumeCloneTask [source computer (unknown name)]-\WindowsBitmapDriverVolumeId=[D8-47-88-B1-00-00-10-00-00-00-00-00] --> [target VM (unknown name)]-8-748(1+0000010000000000 updates, state: 1, percentage: 19, xfer rate (Bps): 7119095

2020-06-16T10:27:16.472+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] VolumeBasedCloneTask source computer (unknown name) --> target VM (unknown name) updates, state: 1, percentage: 19, xfer rate (Bps): 7119095

2020-06-16T10:27:16.472+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] CloneTask updates, state: 1, percentage: 19, xfer rate (Bps): 7119095

2020-06-16T10:27:35.757+01:00 warning vmware-converter-agent[02648] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcNetTcpRead: bRead: -1

2020-06-16T10:27:35.757+01:00 warning vmware-converter-agent[02648] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcNet_Recv: requested 264, recevied only 0 bytes

2020-06-16T10:27:35.757+01:00 warning vmware-converter-agent[02648] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcGetMessage: recv failed:

2020-06-16T10:27:35.757+01:00 warning vmware-converter-agent[02648] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcFssrvr_IO: failed to receive io reply

2020-06-16T10:27:35.757+01:00 info vmware-converter-agent[02648] [Originator@6876 sub=Default] Sysimgbase_DiskLib_Write failed with 'NBD_ERR_NETWORK_CONNECT' (error code:2338)

2020-06-16T10:27:35.757+01:00 error vmware-converter-agent[02648] [Originator@6876 sub=task-3] TargetVolumeWriter::WriteToVolume(): Error (type: 1, code: 2338) writing 655360 bytes to the destination volume at offset 0x00000006e5788000

2020-06-16T10:27:35.757+01:00 error vmware-converter-agent[02648] [Originator@6876 sub=task-3] Converter::BackgroundWriter::WriteToVolume: Write failed with error 37409.

2020-06-16T10:27:35.757+01:00 info vmware-converter-agent[02648] [Originator@6876 sub=task-3] BackgroundWriter::ThreadFunc: Broadcasting "ThreadDone" condition.

2020-06-16T10:27:35.757+01:00 info vmware-converter-agent[02648] [Originator@6876 sub=ThreadPool] Thread delisted

2020-06-16T10:27:35.757+01:00 error vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::QueueItem: CircularBuffer AddItem error, destroy workItem!

2020-06-16T10:27:35.757+01:00 error vmware-converter-agent[05556] [Originator@6876 sub=task-3] BlockLevelVolumeCloneMgr::CloneVolume: Detected a write error during the cloning of volume \WindowsBitmapDriverVolumeId=[D8-47-88-B1-00-00-10-00-00-00-00-00]. Error: 37409 (type: 1, code: 2338)

2020-06-16T10:27:35.757+01:00 error vmware-converter-agent[05556] [Originator@6876 sub=task-3] BlockLevelVolumeCloneMgr::CloneVolume: failed with exception BlockLevelVolumeCloneMgr::CloneVolume: Detected a write error during the cloning of volume \WindowsBitmapDriverVolumeId=[D8-47-88-B1-00-00-10-00-00-00-00-00]. Error: 37409 (type: 1, code: 2338)

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] BackgroundWriter::~BackgroundWriter: Emptying CircularBuffer with abort conditions

2020-06-16T10:27:35.773+01:00 error vmware-converter-agent[05556] [Originator@6876 sub=task-3] ScopedBackgroundWriter::~ScopedBackgroundWriter: Error in ReleaseBackgroundWriter 37409.

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] [StreamCacheWriter] - TotalZeros : 0, TotalWrites : 46187

-->

2020-06-16T10:27:35.773+01:00 error vmware-converter-agent[05556] [Originator@6876 sub=task-3] SingleVolumeCloneTask:DoRun: Volume cloning failed with clone error BlockLevelVolumeCloneMgr::CloneVolume: Detected a write error during the cloning of volume \WindowsBitmapDriverVolumeId=[D8-47-88-B1-00-00-10-00-00-00-00-00]. Error: 37409 (type: 1, code: 2338)

2020-06-16T10:27:35.773+01:00 error vmware-converter-agent[05556] [Originator@6876 sub=task-3] TaskImpl has failed with MethodFault::Exception: converter.fault.CloneFault

2020-06-16T10:27:35.773+01:00 info vmware-converter-agent[05556] [Originator@6876 sub=task-3] SingleVolumeCloneTask [source computer (unknown name)]-\WindowsBitmapDriverVolumeId=[D8-47-88-B1-00-00-10-00-00-00-00-00] --> [target VM (unknown name)]-8-748(1+0000010000000000 updates, state: 4, percentage: 19, xfer rate (Bps): 7119095

2020-06-16T10:27:36.214+01:00 info vmware-converter-agent[02416] [Originator@6876 sub=task-3] CompositeTask canceling #1 subtasks

2020-06-16T10:27:36.214+01:00 info vmware-converter-agent[02416] [Originator@6876 sub=task-3] TaskImpl issuing Cancel command

2020-06-16T10:27:36.214+01:00 info vmware-converter-agent[02416] [Originator@6876 sub=task-3] SingleVolumeCloneTask [source computer (unknown name)]-\WindowsBitmapDriverVolumeId=[D8-47-88-B1-00-00-10-00-00-00-00-00] --> [target VM (unknown name)]-8-748(1+0000010000000000 updates, state: 4, percentage: 19, xfer rate (Bps): 7119095

2020-06-16T10:27:36.214+01:00 warning vmware-converter-agent[02416] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcNetTcpWrite: bWritten: -1

2020-06-16T10:27:36.214+01:00 warning vmware-converter-agent[02416] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcSendMessage: send failed: NFC_NETWORK_ERROR

2020-06-16T10:27:36.214+01:00 warning vmware-converter-agent[02416] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcFssrvr_Close: failed to send close message

2020-06-16T10:27:36.214+01:00 warning vmware-converter-agent[02416] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcNetTcpWrite: bWritten: -1

2020-06-16T10:27:36.214+01:00 warning vmware-converter-agent[02416] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcSendMessage: send failed: NFC_NETWORK_ERROR

2020-06-16T10:27:36.229+01:00 info vmware-converter-agent[02416] [Originator@6876 sub=task-3] VolumeBasedCloneTask source computer (unknown name) --> target VM (unknown name) updates, state: 4, percentage: 19, xfer rate (Bps): 7119095

2020-06-16T10:27:36.229+01:00 info vmware-converter-agent[02416] [Originator@6876 sub=task-3] VolumeBasedCloneTask failed

Any help would be much appreciated!

Thanks

0 Kudos