How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file?

[franl]

I understand that I can manually clone an ESX Server virtual machine disk using vmkfstools -i option and then create a new virtual machine that references that new disk file. But how is that different from just copying the .vmdk files using the Console OS's cp command?

From the vmkfstools man page, it seems like the -i option is mainly for use in combination with the -d option to change the format of the disk file. But I don't want to change the format, just make a copy of it.

Am I violation some fundamental rule of VMware by just copying the .vmdk files instead of using "vmkfstools -i old.vmdk new.vmdk?

[esiebert7625 · Accepted solution]

See this thread...

Using vmkfstools instead of scp to copy virtual disks -

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

Thanks, Eric

Visit my website:

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

[sbeaver]

A while back ago there was some talk about this and if my memory serves me correctly it had to do with the way it does the copy and something about the locks or scsi reservations but it as been a while. Bottom line is you want to use vmkfstools to do this. Maybe someone with better memory will pipe up

Steve Beaver

VMTN Forum Moderator

*Virtualization is a journey, not a project.*

[Dave_Mishchenko]

vmkfstools will allocate all of the disk space required for the copy of the file up front. This is different from cp in that it will keep expanding the size of the copied file as it copies data over. The task of expanding a file on vmfs can create performance issues because the host has to lock the entire VMFS while it expands the vmdk which creates a momentary I/O pause for any VMs on that same LUN.

[troberts]

You can use vmkfstools -i to back up virtual disk files created with ESX Server to non-VMFS file systems, or to convert ESX Server virtual disks to a format that can be used with GSX Server and Workstation. Just using cp to copy an vmdk file to a non-VMFS filesystem will corrupt the vmdk because the blosk sizes are different.

[franl]

troberts wrote: "Just using cp to copy an vmdk file to a non-VMFS filesystem will corrupt the vmdk because the blosk sizes are different." (Emphasis mine.)

But in thread http://communities.vmware.com/message/810197, user wila wrote (emphasis mine):

You can use the cp command to get your data out of the VMFS and it will work -
as you say - provided that the destination filesystem is capable of storing
these large files. For best compatibility however, it is recommended to use the
vmkfstools command to export your disks to a "thin" format of your disks so that
they never cross the 2Gb boundary.
Then for importing the disks back into your vmfs storage, you can again use cp,
however this has downsides as:
- there will be a larger number of SCSI locks which can cause problems with your
other VMs
- the destination VM disk can be fragmented as cp doesn't allocate the complete
file before starting the copy
So in conclusion, the standard unix copy commands work with the VMFS filesystem,
but they are not the optimal way of using your resources.

So does using /bin/cp to copy a vmdk file to a non-VMFS filesystem really corrupt the vmdk file? I understand the performance issues, but I'm concerned about whether or not using cp (or ftp or scp) to copy a vmdk file to a non-VMFS filesystem will corrupt the file.

[oreeh]

Since troberts has the 3-boxes logo and is a VMware TSE...

[franl]

Since troberts has the 3-boxes logo and is a VMware TSE...

Yes, but the Virtual Machine Backup Guide for ESX Server 3.0.1 says on page 14: You can use ftp, scp, and cp commands for copying files to and from a VMFS volume as long as the host file system supports these large files.

So I'm very confused.

[sbeaver]

as the host file system supports these large files

This is what is key. If you try to send a 500GB file to a file systems that does not support files that big or has a different block size things will get screwed up.

[franl]

as the host file system supports these large files

This is what is key. If you try to send a 500GB file to a file systems that does not support files that big or has a different block size things will get screwed up.<div>

</div>

I can accept that, but, seriously, who would read "supports these large files" as meaning "has the same block size as the origin VMFS filesystem"?

[sbeaver]

For me it was getting bit and learning from there

[troberts]

Please read

VMware does not support the use of virtual machine disk files moved from a VMFS volume to a non-VMFS file system by SCP or FTP. When you move a virtual machine disk file to a non-VMFS file system without using the vmkfstools export command or the file manager in the VMware Management Interface, the resulting disk file is completely corrupted and is rendered unusable. This is because the file format is completely different and a non-VMFS file system does not understand how to handle a file created under a VMFS file system.

[oreeh]

Thanks for the KB entry Todd - bookmarked for further reference when the question comes up again :smileygrin:

[franl]

Please read

VMware does not support the use of virtual machine disk files moved from a VMFS volume to a non-VMFS file system by SCP or FTP. When you move a virtual machine disk file to a non-VMFS file system without using the vmkfstools export command or the file manager in the VMware Management Interface, the resulting disk file is completely corrupted and is rendered unusable. This is because the file format is completely different and a non-VMFS file system does not understand how to handle a file created under a VMFS file system.

troberts, thanks for such a definitive statement!

Someone at VMware should clarify the somewhat misleading statement on page 14 of the Virtual Machine Backup Guide for ESX Server 3.0.1.

[troberts]

I believe your referring to the following

"From the service console, you can view and manipulate files in the /vmfs/volumes

directory in mounted VMFS volumes with ordinary file commands, such as ls and cp.

Although mounted VMFS volumes might appear similar to any other file system, such

as ext3, VMFS is primarily intended to store large files, such as disk images with the

size of up to 2TB. You can use ftp, scp, and cp commands for copying files to and from

a VMFS volume as long as the host file system supports these large files."

I should clarify my statement a bit. If you're using ESX3 you can use standard *nix commands like ftp, scp, and cp commands for copying files, but it's is a very inefficient way of copying/moving files. vmkfstools is much better at this and is always the recommend way.

ftp, scp, and cp commands operate in the following manner

1. program says "i need a block to write data"

2. block is given

3.data is written

4. repeat until all data is copied

scsi reservations may occur during this process

vmkfstools operates in the following manner

1. I need X blocks

2. X blocks are given

2. data is written

[wila]

Eeks, i'm being quoted for a technique i do not recommend... please use vmkfstools to export the files first and then use cp/scp/ftp whatever to move the exported files of the vmfs system.

Alternatively if your ESX host has enough non VMFS space, you can use vmkfstools to export directly to the non VMFS filesystem. This last thing is what i normally do myself.

[franl]

Eeks, i'm being quoted for a technique i do not recommend...

wila, I apologize for making it appear you were recommending the wrong way of doing it.

troberts made two very different claims: 1) Using cp/ftp/scp corrupts the copied virtual disk. 2) Using cp/ftp/scp is inefficient and leads to a performance degradation.

This is what confused me. I think I understand the full issue now, and I thank everyone for taking the time to respond to this thread.

[troberts]

Please understand that using cp/ftp/scp to move a vmdk from a vmfs to a non-vmfs filesystem will corrupt vmdk files on a vmfs2 filesystem.

[wila]

Hi frani,

No worries, i'm very vocal most of the times (maybe a bit too vocal) so will complain if i disagree. :smileygrin:

As for your question, as you see from the answers this is one of the areas in which complete and clear documentation is missing. My reply, especially the details, is mostly repeating the quotes of another Vmware TSE (wish i remembered who that was) who tried to explain it before.

Even the answer from troberts with the KB article is (was?) incomplete on some details, like that it is a problem on vmfs2, but not so when using vmfs3. (or click on the related article where it is discussed that the linux ext2 filesystem cannot hold more as 2GB sized files? In reality it 2-64 TB)

Hopefully that documentation gets a little edit now to bring it closer to today and not stating facts from years ago.

So thank YOU for asking.