VMware Communities: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying

Up to Discussions in VI: ESX 3.5

This Question is Answered

1 "helpful" answer available (6 pts)

1 2 Previous Next

18 Replies Last post: Dec 5, 2007 9:34 AM by wila

How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file?

Oct 22, 2007 11:37 AM

franl 9 posts since
Oct 22, 2007

I understand that I can manually clone an ESX Server virtual machine disk using vmkfstools -i option and then create a new virtual machine that references that new disk file. But how is that different from just copying the .vmdk files using the Console OS's cp command?

From the vmkfstools man page, it seems like the -i option is mainly for use in combination with the -d option to change the format of the disk file. But I don't want to change the format, just make a copy of it.

Am I violation some fundamental rule of VMware by just copying the .vmdk files instead of using "*vmkfstools -i old.vmdk new.vmdk*?

Tags: vmdk, clone

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Oct 22, 2007 11:44 AM

esiebert7625 6,786 posts since
Oct 23, 2006 Moderator

See this thread...

Using vmkfstools instead of scp to copy virtual disks - http://www.vmware.com/community/thread.jspa?messageID=704200

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-
Thanks, Eric
Visit my website: http://vmware-land.com/
-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Oct 22, 2007 11:45 AM

sbeaver 7,667 posts since
Nov 1, 2004 Moderator

A while back ago there was some talk about this and if my memory serves me correctly it had to do with the way it does the copy and something about the locks or scsi reservations but it as been a while. Bottom line is you want to use vmkfstools to do this. Maybe someone with better memory will pipe up

Steve Beaver
VMTN Forum Moderator

**Virtualization is a journey, not a project.**

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Oct 22, 2007 11:49 AM

Dave.Mishchenko 8,439 posts since
Nov 15, 2005 Moderator

vmkfstools will allocate all of the disk space required for the copy of the file up front. This is different from cp in that it will keep expanding the size of the copied file as it copies data over. The task of expanding a file on vmfs can create performance issues because the host has to lock the entire VMFS while it expands the vmdk which creates a momentary I/O pause for any VMs on that same LUN.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Oct 22, 2007 8:45 PM

troberts 62 posts since
Nov 3, 2004 VMware

You can use vmkfstools -i to back up virtual disk files created with ESX Server to non-VMFS file systems, or to convert ESX Server virtual disks to a format that can be used with GSX Server and Workstation. Just using cp to copy an vmdk file to a non-VMFS filesystem will corrupt the vmdk because the blosk sizes are different.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 11:24 AM

in response to: troberts

franl 9 posts since
Oct 22, 2007

troberts wrote: "Just using cp to copy an vmdk file to a non-VMFS filesystem will corrupt the vmdk because the blosk sizes are different." (Emphasis mine.)

But in thread http://communities.vmware.com/message/810197, user wila wrote (emphasis mine):

You can use the cp command to get your data out of the VMFS and it will work -
as you say - provided that the destination filesystem is capable of storing
these large files. For best compatibility however, it is recommended to use the
vmkfstools command to export your disks to a "thin" format of your disks so that
they never cross the 2Gb boundary.
Then for importing the disks back into your vmfs storage, you can again use cp,
however this has downsides as:
- there will be a larger number of SCSI locks which can cause problems with your
other VMs
- the destination VM disk can be fragmented as cp doesn't allocate the complete
file before starting the copy
So in conclusion, the standard unix copy commands work with the VMFS filesystem,
but they are not the optimal way of using your resources.

So does using /bin/cp to copy a vmdk file to a non-VMFS filesystem really corrupt the vmdk file? I understand the performance issues, but I'm concerned about whether or not using cp (or ftp or scp) to copy a vmdk file to a non-VMFS filesystem will corrupt the file.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 11:26 AM

in response to: franl

oreeh 9,804 posts since
Nov 30, 2005 Moderator

Since troberts has the 3-boxes logo and is a VMware TSE...

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 11:59 AM

in response to: oreeh

franl 9 posts since
Oct 22, 2007

oreeh wrote:Since troberts has the 3-boxes logo and is a VMware TSE...

Yes, but the Virtual Machine Backup Guide for ESX Server 3.0.1 says on page 14: You can use ftp, scp, and cp commands for copying files to and from a VMFS volume as long as the host file system supports these large files.

So I'm very confused.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 12:09 PM

in response to: franl

sbeaver 7,667 posts since
Nov 1, 2004 Moderator

as the host file system supports these large files

This is what is key. If you try to send a 500GB file to a file systems that does not support files that big or has a different block size things will get screwed up.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 12:16 PM

in response to: sbeaver

franl 9 posts since
Oct 22, 2007

sbeaver wrote:as the host file system supports these large files

This is what is key. If you try to send a 500GB file to a file systems that does not support files that big or has a different block size things will get screwed up.

I can accept that, but, seriously, who would read "supports these large files" as meaning "has the same block size as the origin VMFS filesystem"?

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 12:19 PM

in response to: franl

sbeaver 7,667 posts since
Nov 1, 2004 Moderator

For me it was getting bit and learning from there

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 1:08 PM

in response to: franl

troberts 62 posts since
Nov 3, 2004 VMware

Please read http://kb.vmware.com/kb/900

VMware does not support the use of virtual machine disk files moved from a VMFS volume to a non-VMFS file system by SCP or FTP. When you move a virtual machine disk file to a non-VMFS file system without using the vmkfstools export command or the file manager in the VMware Management Interface, the resulting disk file is completely corrupted and is rendered unusable. This is because the file format is completely different and a non-VMFS file system does not understand how to handle a file created under a VMFS file system.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 1:11 PM

in response to: troberts

oreeh 9,804 posts since
Nov 30, 2005 Moderator

Thanks for the KB entry Todd - bookmarked for further reference when the question comes up again

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 1:16 PM

in response to: troberts

franl 9 posts since
Oct 22, 2007

troberts wrote:
Please read http://kb.vmware.com/kb/900

troberts, thanks for such a definitive statement!

Someone at VMware should clarify the somewhat misleading statement on page 14 of the Virtual Machine Backup Guide for ESX Server 3.0.1.

Reply Re: How is "vmkfstools -i old.vmdk new.vmdk" different from just copying the .vmdk file? Dec 4, 2007 1:37 PM

in response to: franl

troberts 62 posts since
Nov 3, 2004 VMware

I believe your referring to the following

"From the service console, you can view and manipulate files in the /vmfs/volumes
directory in mounted VMFS volumes with ordinary file commands, such as ls and cp.
Although mounted VMFS volumes might appear similar to any other file system, such
as ext3, VMFS is primarily intended to store large files, such as disk images with the
size of up to 2TB. You can use ftp, scp, and cp commands for copying files to and from
a VMFS volume as long as the host file system supports these large files."

I should clarify my statement a bit. If you're using ESX3 you can use standard *nix commands like ftp, scp, and cp commands for copying files, but it's is a very inefficient way of copying/moving files. vmkfstools is much better at this and is always the recommend way.

ftp, scp, and cp commands operate in the following manner

1. program says "i need a block to write data"

2. block is given

3.data is written

4. repeat until all data is copied

scsi reservations may occur during this process

vmkfstools operates in the following manner

1. I need X blocks

2. X blocks are given

2. data is written

1 2 Previous Next

Actions

View as PDF

Up to Discussions in VI: ESX 3.5