Pjokela
Contributor
Contributor

Converter issues with ESXi 6.5 - exporting fails, converter fails

Community, be greeted.

A short backstory. I recently converter 120+ VM's from ESXi 4.1 (running clustered with vCenter) to ESXi 6.5 standalone host (standard license).

These conversions were made with vmware converter and all went by with flying colors. Now the issue is that now I'm totally unable to convert anything from the new 6.5 host. This is a must-need-feature as our users constantly move these VM's from the host to their player/workstation infrastructure.

1) First error is with the converter. Some VM's convert okay but most of them give the "Not enough permissions" at 98%

2) Second error is with the export function in ESXi 6.5 (okay, a bit off-topic). Export goes pretty far but eventually gives network/time-out error.

3) Third error is with the newest converter 6.2. This version gives me the "cannot lock disks" error straight up. It doesn't matter, whether I'm using a NFS share or trying to convert to my local disk.

So far, it might have something to do with the ctb being enabled. Permissions should be okay as I'm having the issues with full root permissions.

Any clues to any of the issues mentioned above? All help is highly appreciated. Currently the only option for moving the VM's is through sftp and it doesn't sound like a plan for tomorrow Smiley Happy

Cheers,

Pekka

0 Kudos
31 Replies
RaspyBear
Enthusiast
Enthusiast

1) First error is with the converter. Some VM's convert okay but most of them give the "Not enough permissions" at 98%

You might try to use other converter for this, in my case I'm always using Starwind V2V no issues yet. Also it's free.

StarWind V2V Converter - Free Tool from StarWind!

0 Kudos
Pjokela
Contributor
Contributor

Okay, here's where we stand at the moment:

I've tested with VMs that had CTB disabled. One conversion went through and the next one gave the 98% "permissions denied" error.

So far there seems to be no clear pattern about the conversions succeeding or failing.

I'll keep this thread updated as my tests go by.

0 Kudos
POCEH
VMware Employee
VMware Employee

This is strange - if the error is again "cant open xxxx.ctk file" then check if these are orphaned files.

0 Kudos
Pjokela
Contributor
Contributor

It's been a while. I'm still having the issue and going to bring this topic active again.

How do I check the possibly orphaned files?

Today I got a report from a user that he got the conversion stuck at 98% with "network error"

Here's a sample from the logs:

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcNetTcpWrite: bWritten: -1

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcFile_SendMessage: data send failed:

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] [NFC ERROR] The operation experienced a network error -- Failed to send data message

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcSendFile: Failed to send block track file 'C:\Virtual machines\SASKE -migration test\xServer2008R2+SQL2014+VS2012Pro-ctk.vmdk': 3

2018-03-27T13:25:10.601+03:00 error vmware-converter-worker[01880] [Originator@6876 sub=task-2] Sysimgbase_Nfc_PutFile failed in H2MDiskCloneMgr with nfcError 'The operation experienced a network error' (error code:3)

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] SSL: Unknown SSL Error

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcNetTcpWrite: bWritten: -1

2018-03-27T13:25:10.601+03:00 warning vmware-converter-worker[01880] [Originator@6876 sub=Default] [,0] [NFC ERROR] NfcSendMessage: send failed: NFC_NETWORK_ERROR

2018-03-27T13:25:10.601+03:00 error vmware-converter-worker[01880] [Originator@6876 sub=task-2] hosted to managed disk clone failed: The operation experienced a network error

2018-03-27T13:25:10.601+03:00 error vmware-converter-worker[01880] [Originator@6876 sub=task-2] Disk clone failed with clone error The operation experienced a network error

2018-03-27T13:25:10.601+03:00 error vmware-converter-worker[01880] [Originator@6876 sub=task-2] TaskImpl has failed with MethodFault::Exception: converter.fault.CloneFault

0 Kudos
POCEH
VMware Employee
VMware Employee

I have no clear idea about this situation but once ctk files are present locally - the question is how they arrive - through Converter's copy or from 3rd party software running locally?

Otherwise, if CBT is disabled, no ctk files should exist locally (must be confirmed with tests), and the pattern seems to be "*-ctk.vmdk", however is these files are transferred from Converter's copy and then if they can't be pushed back - check the file's rights with cacls command or via file-explorer, normally all files for one VM i.e. VMX, VMDK, NVRAM, etc, must be with same access rights.

HTH

0 Kudos
Pjokela
Contributor
Contributor

I understand. This is a bit messy issue or probably a collection of several issues.

One of the users confirmed that when converting smaller VM's the network error doesn't appear and conversion is completed.

About the ctk-files. Is there a way to remove them automatically or via PowerCLI script etc.? Disabling the CTB is not removing anything automatically.

Thanks.

0 Kudos
POCEH
VMware Employee
VMware Employee

For ctk (probably orphaned) files you need to consult with VC/ESX documentation or VC/ESX knowledge base.

I'm not advocating for ctk files removal - this could put machines in inconsistent or unbootable state... (which one is worst?) We can't ensure that some information about these files is not added in VMX or VMDK files, so simple delete of such files could be destructive operation.

From my experience with VC/ESX: if you change VMX file directly - the machine must be removed from inventory and then added again, to consume changes, so disabling CTB could be not so easy.

HTH

0 Kudos
Pjokela
Contributor
Contributor

Thanks for the answers. I've done more tests which all failed (if ctk files are present).

I cloned one system for testing purposes and to play around with:

- unregister/register

- manually deleted ctk files (worked okay but the files came back after starting the VM)

- using converter version 6.2

I'm still quite confused about getting the "not enough permissions". I have pretty much all the permissions available to the host.

Here's a few rows from the latest conversion:

2018-04-05T09:38:32.851+03:00 warning vmware-converter-worker[00752] [Originator@6876 sub=Default] [,0] [NFC ERROR] Nfc_GetFile: Failed to get block track file '[SAS_LUN1] MILANSC18NEW-CLONE\Milan - SC18 - NEW-ctk.vmdk': 11

2018-04-05T09:38:32.851+03:00 error vmware-converter-worker[00752] [Originator@6876 sub=task-21] Sysimgbase_Nfc_GetFile failed in M2HDiskCloneMgr with nfcError 'The session does not have the required permissions' (error code:11)

2018-04-05T09:38:32.851+03:00 error vmware-converter-worker[00752] [Originator@6876 sub=task-21] managed to hosted disk clone failed: The session does not have the required permissions

2018-04-05T09:38:32.866+03:00 error vmware-converter-worker[00752] [Originator@6876 sub=task-21] Disk clone failed with clone error The session does not have the required permissions

2018-04-05T09:38:32.866+03:00 error vmware-converter-worker[00752] [Originator@6876 sub=task-21] TaskImpl has failed with MethodFault::Exception: converter.fault.CloneFault

0 Kudos
POCEH
VMware Employee
VMware Employee

Having ctk files back when starting VM means that CBT is not disabled. I'm not sure is it possible to turn off CBT per VM or per Host. Have you follow the described procedure for disabling CBT?

About permissions - try to log as user 'root' if still see this message - then there is meta data written somewhere which disable access to files, if not see that message again, then check users and privileges defined for host, may be Veritas Backup creates special user(s) and role(s) to access these files.

HTH

0 Kudos
Pjokela
Contributor
Contributor

Actually I didn't disable them for this test VM before re-registering it. It's just that I've never been required with doing such thing. I mean that with the old 4.x version and vCenter this wasn't an issue.

Nevertheless, do you know if I could disable CBT for all VM's at once?

0 Kudos
Pjokela
Contributor
Contributor

A quick edit. I found the KB2139574 which has a pre-made PowerCli script for resetting cbt on multiple VM's. I'll keep this thread updated as I run more tests.

0 Kudos
JonasMR
Enthusiast
Enthusiast

Hi @Pjokela 

I had the same issue, you need to reset CBT on a vSphere virtual machine, you can do this https://kb.vmware.com/s/article/2139574

it's important the last step "start the virtual machine", then I stopped the MV and I could clone.

best regards

0 Kudos