how fast to copy files on vmfs ?

[alex777]

Hello.

ESX 3.0.1, ESX 3.5 with local scsi and sas storages (vmfs 3x).

1) I try to copy(by MC, Veeam fastSCP and cp command) any files (size > 1GB) from one local disk to another on not loaded server (all VMs are switched off).

The maximal speed of copying was < 15 MB/S.

2) I try to copy(by MC, Veeam fastSCP and cp command) any files (size > 1GB) from local disk (datastorage) to shared folder (SMB and NFS) on Win2k3 server (Another Physical server) (all VMs are switched off).

The maximal speed of copying was < 3 MB/S.

Why speed of copying such slow?

How most effectively to copy files on VMFS?

How most effectively to copy files with VMFS on other server under win2x (NTFS)?

[mike_laspina]

Hi,

The COS has limited resource shares and will not be able to run at the maximum.

[oreeh]

Using "normal" tools (cp, mv, ftp, scp) to copy to a VMFS volume is a bad idea.

These tools allocate the required space on demand which could lead to fragmentation and therefore to bad performance.

For VMDKs always use vmkfstools to copy / clone a disk.

[sbeaver]

That and there is a limit put in place for the write speed of the VMFS partitions

Steve Beaver

VMware Communities User Moderator

====

Co-Author of "VMware ESX Essentials in the Virtual Data Center"

Coming soon to a store near you!

*Virtualization is a journey, not a project.*

[alex777]

What for such limit is created?

[alex777]

How by vmkfstools to copy vm disk file to another storage ?

[oreeh]

If the host has access to both storages:

vmkfstools -i path_to_src_vmdk path_to_dst_vmdk

If the host doesn't have access to both storages:

export the VMDK to an ext3 partition, copy the files to the other host onto an ext3 partition, import the VMDK to the datastore

[alex777]

root@vm6 /# vmkfstools -i /vmfs/volumes/st2/vm2/vm2-flat.vmdk /vmfs/volumes/vm6:storage1/dest.vmdk

DiskLib_Check() failed for source disk The file specified is not a virtual disk (15).

sorry

vmkfstools -i /vmfs/volumes/st2/vm2/vm2.vmdk /vmfs/volumes/vm6:storage1/dest.vmdk

works well.

[oreeh]

Should be faster as well

[alex777]

Whether there are special tools for copying any files (not vmdk) on vmfs?

And how most effectively to copy files with VMFS on other server under win2x (NTFS)? How most effectively backup VMs in storage on Win2x server ?

[oreeh]

No. For the other files the fragmentation isn't a problem.

Logs, VMSD and VMSN files are growing anyway, VSWP files don't need to be copied and the VMX file is to short to get fragmented.

[alex777]

And if need to copy vmdk file with shared folder on win2x to vmfs?

Thanks for you answers !

[oreeh]

ESX doesn't support shared folders.

If you mean to import a VMDK from WS / Server / Player / Fusion:

either convert it to the 2gbsparse format, copy it to the an ex3 partition, import is using vmkfstools

or use VMware Converter

[alex777]

For example i have some vmdk files on win2k3. And I need to copy

quickly them on ESX.

Earlier I did shared folder on win2k3. Then

mounted (mount -t smb) it on ESX server. Then used mc or cp for copying vmdk from mounted folder to vmfs.

Or used FastSCP.

[oreeh]

You can import the VMDK from the mounted folder using vmkfstools

[alex777]

Yes. It work. But speed is limeted by smb (&lt; 4 MB/S)

It would be desirable fast to copy vmdk files

with win2k on ESX through LAN.

How on yours correctly to do backup VMs

if the backup server on win2k? Now I do backup by vcbMounter,

where mountpoint is shared folder on win2k3. But it borrows a lot of

time, as speed of copying through smb &lt;3 MB/S .

[wila]

You should * not * treat the COS as if it is a normal linux system.

It is tuned to manage your ESX server and this is also why -for example- the speed of normal file operations like a "cp" and things are throttled.

If you overload your COS with these type of actions, ALL of your VMs will suffer, you don't want all of your users start complaining to you just because you tried copying some files or would you?

Down below in the thread you also speak of using your COS to share files. Beware that this has security implications, you should not have your COS share files (or access an SMB share) as this means your administrative network and VM network are no longer isolated networks.

If you are in need for speed on copying files, you will see that copying files up/down using a real VM will be much faster as it is not as severely throttled as the COS is in this respect. If the target is the VMFS then VMware converter is likely to be the fastest solution as it copies the VM straight into VMFS using the vmkfstools mechanism (so no risk on fragmentation)

--

Wil

[mike_laspina]

In addtion to the COS limits of mem and disk are the CPU. COS only runs on one cpu so you can create serious context switching with the other threads on the host if you load it down to heavy.

[alex777]

Thanks for your answer !

I try to copy vmdk files from / to vmfs by datasote browser. It works well.

Yet it would be desirable to find out optimum strategy of reserve copy on win2k backup server.

I will open the proper theme.