Moving VMs from one ESXi server to another - what a pain!!!

[ShahidSheikh]

This is where I miss my setup with VMware Server. I do not have a iSCSI or NFS central storage. Before I started the move to ESXi, all my VMware Server (1.0.7) machines were Ubuntu 7.0.4 servers all of which had the directories where VMs were stored exported through NFS and cross mounted on each of the servers. So on each server I had the directory structure:

/vmfs/svm01

/vmfs/svm02

/vmfs/svm03

/vmfs/svm04

Where only one directory was local and the remaining three were NFS mounts from the three other servers. This made moving moving VMs around a breeze. I could even run vmware-vdiskmanager with input and output VMs being on different servers. The only thing that was slow was provisioning thick vmdks.

Now under ESXi moving VMs is a pain. Apparently scp in ESXi does not support the -C (compression) flag. Through scp, I can either scp thick vmdks from one ESXi to another or import them into thin vmdks and then scp them. Either way its a painfully slow process. And scp only gives a maximum thruput of about 2.5 MBytes/sec (20 Mbps) on my 1 GB connected NICs (VM nic is separate from management nic). All servers are either Dell (2850, 2950) or HP 2U (380 G4) servers, all with 4 or more U320 SCSI drives in a hardware RAID.

I was hoping in RCLI I would be able to run vmkfstools.pl script across machines but alas, it only runs vmkfstools operations on a single machine too.

Right now I am trying to move one of my VMs that has a 120 GB vmdk and it says its going to take 15 hours to move. I can reduce the server downtime significantly (I think) by taking a snapshot and moving that first, then downing the VM and moving the deltas but still all of this for doing something I could do fairly painlessly in VMware Server seems a lot of work.

My question is that does anyone have a better way of moving VMs from one ESXi to another? I will eventually have a centralized NFS store but that is not going to happen for another 2 months.

It would be very cool if there was a way to NFS export the local datastores in ESXi.

[TLF]

Has anyone got rsync daemon to work? I got following error in log:

rsync: getpeername on fd4 failed: Invalid argument (22)
rsync error: error in socket IO (code 10) at clientname.c(168)

[wessie]

Mispost, sorry

[devzero]

nope - same issue here - but i`m unsure if it`s due to the binary from this thread or because of the esxi environment. will compile a binary myself next week and try again.

[devzero]

but you don`t use it as a daemon on esxi, do you?

[TLF]

I started rsync as a daemon on a second ESXi box. It started well, but got that error when trying to connect to it.

[devzero]

the worst thing is: there is an error message in /var/log/messages which seems to be specific to vmware console os kernel. so it will be hard to find out, what it means.

can you post the complete log of such failed connect here? i`m off site and don`t have access to the log.

[birnenschnitzel]

Maybe an FTP server on your ESXi will help to speed things up...

http://communities.vmware.com/message/1121392

Best regards.

[devzero]

nice link, thanks.

[fharmann]

It's possible to use rsync in daemon mode.

Check this thread: http://communities.vmware.com/message/1273980#1273980 (it has a working non-static rsync 3.0.6 binary for esxi 3.5)

Just remember that you can't start rsync in daemon mode from inside the service console(you will get getpeername errors if you try). It has to be started by inetd (obviously the service console is some kinf of fakeroot shell).

I love the rsyncd solution. Now shadowing 2 esxi works easily and fast. Just replace some lines in the ghettoVCB script to use rsync as target.

And in combination with --write-batch-only or something like backuppc you can even do nice incremental backups of your vm's with rsync,-)

Frank

[cdleighton]

I have used Veeam to copy comr one machine to another, but how do I get the VM into the destination machine's inventory?

- Chris

[cdleighton]

Last year you said:

> and you wont need to turn SSH on

How do you turn on SSH?

Once done can you simply SSH to the machine and wander about like any other Linux/Unix?

- Chris

[kpc]

this has been asked many times already, try the search function or go here:

http://www.run-virtual.com/?p=223

It's Linux but a very sparse implementation. Once SSH is up and running just do the usual SCP or mount an NFS share that both ESXi servers can see - easy

[Risner]

fharmann, you said "I love the rsyncd solution. Now shadowing 2 esxi works easily and fast."

I've been using an rsync I compiled a year ago on all my ESXi's and it is far from fast. From what I could tell ESXi meters the disk I/O of the service console, and this causes rsyncs to take exactly as long as scp's take. So I gained no speed improvement for large (snapshot) vm's.

Can you confirm this is not a problem? Is this a benefit of running it as a daemon from inetd?

[devzero]

fharmann, you said "<span class="jive-thread-reply-body-container">I love the rsyncd solution. Now shadowing 2 esxi works easily and fast."

I've been using an rsync I compiled a year ago on all my ESXi's and it is far from fast. From what I could tell ESXi meters the disk I/O of the service console, and this causes rsyncs to take exactly as long as scp's take.

You did not use rsync in daemon mode but in remote-shell mode, did you?

i assume that you used rsync via ssh and that`s the reason why it was as slow as scp.

[gary0224]

what I do is shrink the disk first and cone the vm through vC converter

adjust the HD again after coning.

[cdleighton]

Yesterday I my question was not so clear:

&gt; I have used Veeam to copy comr one machine to another, but how do I get the VM into the destination machine's inventory?

The directory is on the destination ESXi machine.

I used Veeam Fast SCP, but now I can also use SSH.

I do understand how to make the copyied VM available in the inventory. Neither "import" or "new" can work for me. New can see the virtual disk, but how can I use the .VMX file as the key?.

[scowse]

I love the rsyncd solution. Now shadowing 2 esxi works easily and fast. Just replace some lines in the ghettoVCB script to use rsync as target.

Can anyone elaborate on "Just replace some lines" to get rsync working in ghettoVCB script. I dont need rotation - just snapshots.

[devzero]

>I've been using an rsync I compiled a year ago on all my ESXi's and it is far from fast. From what I could tell ESXi meters the disk I/O of the service console, and this >causes rsyncs to take exactly as long as scp's take. So I gained no speed improvement for large (snapshot) vm's.

the i/o capping is even worse on esx4(i). mind that using ssh as a transport for rsync is also slow, you should use unencrypted, direct rsync connection to rsyncd for fast transfer.

[vanyel]

After copying a vm to a datastore, in the client select the vmhost in the inventory list, then the Configuration tab, and Storage. Right click on the datastore and "Browse Datastore". Find the vmx file of interest, right click and "Add to Inventory".

[wessie]

Bedankt voor u bericht.

Ik kan u helaas niet antwoorden in verband met een vrije week.

Vanaf 21 september ben ik weer aanwezig.

Met vriendelijke groet,

Erik Westdijk