Fastest way to transfer files between servers...

[yougotiger0]

I have a question in regards to what would be the best option to speed up the transfer of files between 2 servers in our VMware environment. We are planning on moving our GroupWise email from netware to windows and upgrade Groupwise (v7 to v8) at the same time. We're anticipating moving around ~125GB of data (~517,000 files) and expect that it will take several hours. We'd like to minimize this time as best we can so we're wondering what would be the best configuration of things to make that happen? Both VM's on the same host? Both VM's on the same DataStore? On different hosts, or datastores? Our configurations are listed below:

-Currently running vSphere 4.1 ESXi

-Data stored on 2 Dell Equalogic 5500 SANs

-One SAN in each of 2 locations, physically about 3 miles apart

-3 Hosts, 2 in one location and 1 in the other (same locations the SANs)

-Source of data transfer is Netware 6.5, destination will be Windows 2008 R2 DataCenter

-Gigabit links between all pieces of our VMware environment, including the link between sites (over fiber).

-All 3 hosts are capable of running all of our VM's without overtaxing them, so if we need to move VM's to another host, it isn't a problem.

We'd like to know what the best way to set everything up for the transfer is so that we can prepare before hand. Any suggestions?

Thanks in advance.

[Mike_Deardurff]

I'm actually very interested to see other users replies to this.  It is a very good question and coincides with my previous forum post regarding slow network speeds between VMs, either on the same or separate hosts.

-Mike

[yougotiger0]

Well, I'm quite sad to see that nobody took up the challenge and the question shall forever be unanswered.

[DCjay]

Hi,

I would use Veeam fastscp.

http://www.veeam.com/vmware-esxi-fastscp.html

[yougotiger0]

Thanks for your recommendation @DCjay. I'm not sure that's exactly what we were looking for (we're done now), but we wanted to transfer files from WITHIN one VM to WITHIN another VM, and know how to setup our environment to make that happen the fastest. Veeam Fastscp seems to copy the VM as a whole over (or am I wrong?)

But thanks for being the only person so far to take a stab at it...

[Sreejesh_D]

Since VM1 and VM2 are located in the same host, i would suggest you to create a clone of  the virtual disk (where 135GB of data saved) from VM 1 and add the new disk to VM2.

[sflanders]

Out of curiosity, why are you looking for the fastest way? You could run NFS on the Windows VM and rsync the data. Not the fastest way, but assuming the issue is switchover time, rsync would allow a quick switchover.

[yougotiger0]

We're just trying to figure out what would be the optimal setup for transfering files from one server (VM) to another. We're not sure if having them on the same SAN would speed it up (becuase VMware call to the SAN and does block copies etc), or if it would be faster to have the VM's on different SAN's due to read/write speed etc. VM's on the same host can transfer between at back plane speeds, but is that the best due to having to transverse the LAN to get to the SAN's anyway?

In this particular example, we were going from different file systems (Novell to NTFS) so we can't just clone the drives etc, we had to copy the data. In the end, we did a kind of Rsync thing, copied one night, and then did a differential copy and our work the next night.

[sflanders · Accepted solution]

Well the fastest way would be to offload the clone/copy operation to the SAN. If your configuration supports VAAI then the clone would happen completely on the SAN assuming the source and destination storage device is the same SAN. If to different SANs then the network factors in no matter what. As for running on the same host, that would be benficial for cloning a VM (primarily the memory footprint) as it would not need to take place over the network.

Hope this helps!

[DSTAVERT]

Veeam fastscp works for transfering files but rsysnc would be my choice as you have used.

[yougotiger0]

VAAI seems like the way to go, I'll have to look into whether our SANs support it, (Equallogic PS5000's)...

That's the kind of info I think we're looking for, If our SAN is VAAI compliant, then put both VMs on the same san, do the VM's need to be on the same host, or can the be on differing hosts and still offload it to the SAN?

[sflanders]

If you are cloning VMs that it makes sense to keep them on the same host to prevent memory footprint from being copied over network. In terms of offloading to SAN, it does not matter as all hosts in the cluster will need to connect to the same SAN and offload is handled on the SAN.

[yougotiger0]

Thanks, we'll look into our SAN and VAAI for the future. Much appreciated.