Vcenter 6.0 converter stalled with operation timed out

[pcroninbda]

Can anyone help with my particular issue. I am trying to convert form a running VM to another ESX server in the same vCenter. It looks like it completely copies and then gets to 91% stalled timing out on something. Below is the vmconverter logs

Task: Convert physical machine.

Starting block-level cloning for volume 'C:

Completed cloning volume 'C:'

Creating a snapshot of the source system

I have to add, the original running VM is originally from VirtualBox, but we have it running on the ESX box but using a NFS share to a storage area (Datto - Backup appliance). The VM spins up with no issues, so we ultimately want to get the NFS share converted to another ESX server, but this has to be done live as the client can not be down for more than a few minutes.

2015-11-20T01:45:32.918-05:00 info vmware-converter-worker[04212] [Originator@6876 sub=task-2] CloneTask: Agent task moid=task-4

2015-11-20T01:45:32.918-05:00 info vmware-converter-worker[04212] [Originator@6876 sub=task-2] Worker InitialSyncCatchUpTask updates, state: 1, percentage: 0, xfer rate (Bps): <unknown>

2015-11-20T01:48:07.189-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T02:15:34.844-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T02:18:07.179-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T02:45:34.827-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T02:48:07.173-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T03:15:34.808-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T03:18:07.161-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T03:45:34.799-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T03:48:07.166-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T04:15:34.792-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T04:18:07.155-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T04:45:34.784-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T04:48:07.151-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T05:15:34.766-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T05:18:07.144-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T05:45:34.765-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T05:48:07.143-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T06:15:34.753-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T06:18:07.128-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T06:45:34.742-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T06:48:07.121-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T07:15:34.732-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T07:18:07.101-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T07:45:34.737-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T07:48:07.097-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T08:15:34.723-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T08:18:07.087-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T08:45:34.719-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T08:48:07.090-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T09:15:34.716-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T09:18:07.095-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T09:45:34.707-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T09:48:07.075-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T10:15:34.700-05:00 warning vmware-converter-worker[04212] [Originator@6876 sub=task-2] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

2015-11-20T10:18:07.064-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

[POCEH]

If the conversion continues, what the problem is?

If the conversion fails then upload log to check them.

[pcroninbda]

The conversion never continues, it's stalled at 91% with the logs putting out this line once in a while:

2015-11-20T10:18:07.064-05:00 warning vmware-converter-worker[06256] [Originator@6876 sub=task-1] [WaitUntilComplete] WaitForUpdates timed out: Operation timed out

It never really fails

[POCEH]

Could you upload log bundle or share details of conversion?

[joshopper]

Have you tried re sizing the disk when you create the conversion job so it does a file level clone instead of block level clone? This used resolve some problem conversions back in the day.

[pcroninbda]

Yes, we have selected Thin Provision, if that's what you mean.

[joshopper]

Nope that is not what I mean. If while setting up the conversion to run you change the size of the c: drive even just a little bit, select the drive and add say 5Gb of space.the converter uses a different methodology to run the conversion and this sometimes clears up some issues.

[pcroninbda]

I just tried that and added an extra 5 GB. It balked at me and said to disable synchronization even though it says it will ne allowed if its NTFS, which it is. I need that turned on to make sure we keep the data up to date when it finishes.

[patanassov]

As you have already noticed, synchronization doesn't work with file level cloning mode; it requires block level.

Synchronization uses a driver that tracks the writes on the source (called bitmap driver) and VSS snapshots. During synchronization the full cloning of the volumes has two parts: first a plain full cloning takes place, then the bitmap drivers starts tracking, then a VSS snapshot is taken, and an incremental cloning is done from that snapshot (in order to guarantee data consistency). This is "InitialSyncCatchUpTask" you see in the log.

It is possible that the NFS share has somehow interfered with the normal working of the bitmap driver (this has never been tested if I am not mistaken).

Please upload the agent logs from the source machine, there might be some hints there. They are in "%ALLUSERSPROFILE%\VMware\VMware vCenter Converter Standalone Agent\logs".

Regards,

Plamen