Increasing the cloning performance

[patanassov]

There have been many complaints about transfer rate degradation in Converter 5.0.

Converter uses NFC (a proprietary VMware protocol) for cloning to managed destination. Security has been enhanced in Converter 5.0 by encrypting the data transfer. Unfortunately this has caused a more severe performance degradation than expected.
Switching off SSL encryption is a way to work around this issue. Here is how it is done:

1. Open the converter-worker.xml configuration file. It is located in "%ALLUSERSPROFILE%\VMware\VMware vCenter Converter Standalone" folder for Windows Vista or newer or in "%ALLUSERSPROFILE%\Application Data\VMware\VMware vCenter Converter Standalone" for older Windows versions.
2. Set the key Config/nfc/useSsl to false and save the configuration file.
3. Restart "VMware vCenter Converter Standalone Worker" service.

I.e. it should look like:

    ...
    <nfc>
       <readTimeoutMs>120000</readTimeoutMs>
       <useSsl>false</useSsl>
    ...

[sparrowangelste]

Thanks,

I ran some tests: http://sparrowangelstechnology.blogspot.com/2012/07/vmware-converter-does-disabling-ssl.html

The first time, using SSL was faster than not! weird.

On the first try for SSL off I got Submit job failed: The specified parameter was not correct: 'spec.synchronizeImmediately'.

2nd try the conversion with SSL off was 40% faster.

so if you are having problems, try a reboot and do it again.

i rebooted between conversions.

all parameters were the same, same guest, host, VMFS, etc.

convert1: ssl on: faster  3x faster than convert2

convert2:ssl off:  much slower

convert3: ssl on:  same as 1
convert4:ssl off:  MUCH faster this time, 40% faster than SSL being true.

In some future p2vs Ill need to do this will come in very useful!

maybe a reboot if you arent getting the performance and throughput desired.

[geMarkMc]

I am out of the office until Monday July 9, 2012.

Thank You,

Mark

[cjreddragon]

I've done some testing too, though I can't seem to get the speeds you're talking about.

I'm converting a single virtual machine, which was created with vmplayer.

when ever I turn off ssl, my transfer rate is around 400kb/sec, as reported by VMC, though network usage is around 40% (100mbit) as reported from taskmanagers network usage.

when I turn ON ssl, transfer rates are around 1000kb/sec (VMC) and network usage is around 80-90%

are you all doing a p2v or are there someone who has done a v2v and had the speed ?

Target server is an ESXi 5.0

VMC is 5.0

Client machine is an Core-i5 @ 3.3GHz using a Sata 6Gbps disk/interface

only slow thing here is the networking, but data throughput is way too slow compared to link speeds.

(I got a clean 100mbit connection to the server)

[sparrowangelste]

My tests were v2v,  ive done some p2v also outside a test environment with similar results.

Do you have converter on the vm player  machine or doing a remote conversion?

weird that you get the opposite result with ssl off.

if your source vm is large and it s taking a long time and you have limited bandwith,

it might be just faster to put the vm on a usb drive,

run a passthrough to a vm on the esxi host that has converter

then convert the vm on the host directly.

obviousy not ideal for large scale deployments but for the one or two p2v/v2v that show up not a bad workaround.

[cjreddragon]

The converter and player is located on the same computer.

I only did a converte vm-file to infrastructure (if that makes any sense)

I did try to copy the VM directly to the ESXi server using scp, which took around 20 min, but the ESXi complained

that it didn't recognize the virtual disk, even though it was created with VMP4.

hmm .. ESXi host with converter installed?  please explain that one 🙂

[sparrowangelste]

the host wouldnt have converter,

a virutal machine on the host would have converter.

[continuum]

> but the ESXi complained

thats harmless - you can import the vmdk to esxi-compatible format with

vmkfstools -i uploaded-vmdk-from-workstation.vmdk esxi-compatible.vmdk

after the import simply exchange the vmdks - test the VM and if it works - delete the uploaded vmdk

[cjreddragon]

thats interesting, gotta try that next time, thanks 🙂

@sparrow, now I get what you mean. thanks

[bwhudgins]

Just to chime in on the original post.  After 14 hours converting a VM from 100GB to 50GB drive using about 17GB of storage and from ESXi 4.1 to ESXi 5 I was at 98% with the average speed at 98KB/sec.  I made the change and I am now running over 20MBps and the process finished within 20 minutes.  Thank you to all of the smarter people than I!!!

[p14men]

Thank you very much!

I increased it from 750Kbs to 75Mbs. Which is 100 times faster!

I'm on Windows7 and the file is located at

C:\ProgramData\Application Data\VMware\VMware vCenter Converter Standalone\converter-worker.xml

[Asteroza]

Converter 5.0.1 still has this set to true by default...

[Desparado9]

Excellent tip. My 2.5Tb P2V took 16hrs instead of 2+ days. Avg ~30Mbps. ( KB: 2020517 ) Thanks!!

[rootlevel]

Thank you so so much for this. I've gone from 580KB/s - 1.9MB/s transfers to 50+MB/s. 4 days transfer down to 7hrs. Thank you again!

[patanassov]

I'm glad it helped.

However that much difference seems strange (I'd expect 2-3 times increase at most). Could you share something about your hardware - e.g. is the source machine using an old/slow processor, is it heavily loaded? Same for the target ESX (or WS)?

Thanks,

Plamen

[rootlevel]

Sure, the least I can do. Dell R510's (2x with same issue) each with 2x 4-way xeon 2.4GHz and 24GB 1333 ecc ram. Very light load on one, zero load on the other with same transfer problem.

Problem occured during V2V clones (hot or cold) to HP z210's as a temp measure to keep the servers up during primary hardware reconfiguration. The HPs are i7 but give a non-virtualising error during install, all guests are 32bit so not a *huge* issue. As they have to de-SSL the transfer I guess they could've been part of the transfer issue, still, they are beastly 4-way 3.4GHz machines with 16GB 1600 ram so decrypting a simple transfer like this is not beyond their means. They have single-disk datastores, one per guest (only running for a day or two). The guests are performing quite well. Only requirement for the HP customized installer was banging in an intel nic as I'm new to adding esxi drivers...wtf...

One problem I recently became aware of was having esxi 4.1 on the same diskset as the datastore, could be part of the issue however of course that was still the case when the system was able to deliver up to 60MB/s.

Disksets were 4x100GB SSD and 4x900GB all in raid5 - again, not ideal but still able to deliver 60MB/s.

Now installing 5.1 to internal USB-stick with datastores at raid10 with some extra disks, can't wait for the outcome.

Thanks again - the SSL issue still has me reeling. I'm guessing it relates to the same old issue with SCP.

[StevenLancer]

Hello,

is it necessary to reboot the server after editing the converter-worker.xml file? i restarted the services but the speed is stuck at 5MB/sec (when I copy a file over the same network & nics it runs at 50mb/sec.

thanks for your help

[patanassov]

Yes, it is, From the initial post:

"Restart "VMware vCenter Converter Standalone Worker" service."

[KFM]

This is an interesting dilemma.  I'm in the same situation as StevenLancer.

I have modified the required files and restarted the VMware Converter Worker service and have even restarted the server that hosts the VMware Converter Server and I still get pathetically low throughput.

Now here's the catch.  Ever since ESXi was released, I've found the throughput through the management network has been abysmally slow.  Evidently so have other people as witnessed in these threads:  http://communities.vmware.com/thread/427466?start=0&tstart=0#427466 and http://communities.vmware.com/thread/168637?tstart=0.  Whilst the second link refers to ESXi 3.5 I believe the problem is still evident in later releases.

So my question is, if the management network is terribly slow (~6-10 Megabytes/sec) then how is VMware Converter achieving such high throughput rates when you disable NFC?  AFAIK Converter uses the same vmkernel used for the management network to do its transfers through.

To further muddy the waters, I have two clients with similar setups.  One has the management network on a distributed switch and one has the management network on a standard vSwitch.

For the client with the management network on the distributed switch:

• File copy using SCP - ~8 MB/s
• File copy using datastore browser - ~25 MB/s

For the client with the management network on a standard vSwitch:

• File copy using SCP - ~8 MB/s
• File copy using datastore browser - ~8 MB/s

All four file copies use the same NICs attached to the vSwitch (distributed and standard) so I can definitely prove that the NICs assigned for management use are capable of higher throughputs.

I'm at my wits end on how to speed up the throughput over the management network for the second client, specifically when using VMware Converter.  I have a number of P2Vs to perform with large volumes that I need to convert in the weeks coming up.

Any help/ideas would be much appreciated.

[Asteroza]

5.1.0 is also still true by default. It resets this back to true during the upgrade too.

As for slow management network performance, I don't have any experience on paid ESXi, but I was under the general assumption that free ESXi was more or less capped at 10MBps on the management network as form of passive-aggressive bullying to get people to buy ESXi.

[vikramkumarpuli]

Hi,

I am using Standalone converter 5.1. I am not able to edit the Converter-worker.xml to disable the ssl. I get error Access to C:\ProgramData\VMware\VMware vCenter Converter Standalonee\converter-worker.xml denied"

Thanks.