VM image take up more space on my iSCSI than local disk

[idivad]

I am running ESXi, 4.1.0 and have been wanting to upgrade for a while.  Wanting to be careful with my data I purchased a PX4 from Iomega with 12TB of  disk space, on a raid 5 configuration; so I have less than 10TB Available.  I quickly learned that ESX has a 2TB limit for iSCSI datastores, so I created the appropiate sized datastore of 2TB.  I still have space on the px4 that I plan on using for additional datastores and nas storage.

The original (local) datastore on my ESX server is 1.6TB and and about  1TB full.  I copied all my vm folder, using the datastore browser from the original datastore to the new iSCSI datastore located on the PX4,  after overnight transfer, I found the that transfer failed due to out of space on the target datastore.

I did the following analysis.

on the Source datastore (original datastore, copy from)

/vmfs/volumes/4dd40594-5eb7ea82-e6cf-782bcb173743/ubuntu # ls -l
-rw-------    1 root     root         1728262144 May 26  2011 ubuntu-000001-delta.vmdk
-rw-------    1 root     root                316 May 26  2011 ubuntu-000001.vmdk
-rw-------    1 root     root        56237436928 Sep 11 20:36 ubuntu-000004-delta.vmdk
-rw-------    1 root     root                323 Jun  4  2011 ubuntu-000004.vmdk
-rw-------    1 root     root          620965888 Oct  2  2011 ubuntu-000006-delta.vmdk
-rw-------    1 root     root                323 Sep 28  2011 ubuntu-000006.vmdk
-rw-------    1 root     root         2147483648 Sep 28  2011 ubuntu-27ce52bc.vswp
-rw-------    1 root     root              28212 May 26  2011 ubuntu-Snapshot1.vmsn
-rw-------    1 root     root              28281 Sep 27  2011 ubuntu-Snapshot5.vmsn
-rw-------    1 root     root       107374182400 May 26  2011 ubuntu-flat.vmdk
-rw-------    1 root     root               8684 Sep 28  2011 ubuntu.nvram
-rw-------    1 root     root                495 May 26  2011 ubuntu.vmdk
-rw-r--r--    1 root     root                768 Sep 11 20:37 ubuntu.vmsd
-rwx------    1 root     root               2800 Sep 28  2011 ubuntu.vmx
-rw-------    1 root     root                261 Sep 28  2011 ubuntu.vmxf
/vmfs/volumes/4dd40594-5eb7ea82-e6cf-782bcb173743/ubuntu # ds -s ../ubuntu/
-ash: ds: not found
/vmfs/volumes/4dd40594-5eb7ea82-e6cf-782bcb173743/ubuntu # du -sh ../ubuntu
60.9G   ../ubuntu

on the target datastore (iSCSI, copy to)

/vmfs/volumes/504e7239-269865cb-558f-782bcb173741/ubuntu # ls -l
-rw-------    1 root     root         1728262144 Sep 12 05:56 ubuntu-000001-delta.vmdk
-rw-------    1 root     root                316 Sep 12 05:56 ubuntu-000001.vmdk
-rw-------    1 root     root        56237436928 Sep 12 06:51 ubuntu-000004-delta.vmdk
-rw-------    1 root     root                323 Sep 12 06:51 ubuntu-000004.vmdk
-rw-------    1 root     root          620965888 Sep 12 05:54 ubuntu-000006-delta.vmdk
-rw-------    1 root     root                323 Sep 12 05:54 ubuntu-000006.vmdk
-rw-------    1 root     root         2147483648 Sep 12 03:39 ubuntu-27ce52bc.vswp
-rw-------    1 root     root              28212 Sep 12 05:54 ubuntu-Snapshot1.vmsn
-rw-------    1 root     root              28281 Sep 12 05:56 ubuntu-Snapshot5.vmsn
-rw-------    1 root     root       107374182400 Sep 12 05:53 ubuntu-flat.vmdk
-rw-------    1 root     root               8684 Sep 12 06:51 ubuntu.nvram
-rw-------    1 root     root                495 Sep 12 05:53 ubuntu.vmdk
-rw-------    1 root     root                768 Sep 12 03:40 ubuntu.vmsd
-rw-------    1 root     root               2800 Sep 12 03:37 ubuntu.vmx
-rw-------    1 root     root                261 Sep 12 05:56 ubuntu.vmxf
/vmfs/volumes/504e7239-269865cb-558f-782bcb173741/ubuntu # du -sh ../ubuntu
156.6G  ../ubuntu

Can someone please help, I'm not sure what to make of this?  each of the file sizes are identical on source/target, but the folder size (du command) shows differnt disk consumptions.  I'm getting similar behavior with other vm, the increase in size for each vm is what I belive is causing my out of space error.

[continuum]

ubuntu-flat.vmdk is a thin provisioned vmdk on the local disk - a simple copy with Datastorebrowser  to iSCSI inflates this to thick provisioned

use
vmkfstools -i <localdisk>/ubuntu.vmdk <iscsi>ubuntu.vmdk -d thin
to prevent this

[idivad]

thanks for your help.  after running the command, I noticed that it only created a single vmdk, and did not carry over the snapshots.  is there a command that can copy over (for backup purpose) all the snapshots

[continuum]

you said that you already copied the snapshots ???

why do you want to copy them again ?

[idivad]

sorry I must have misunderstood to use the vmfkstools command in lieu of the copy command.  so to recap. if I want to make a copy of each vm including its snapshots; and not over consume disk space on my target datastore I should do the following?

1. Turn off VM.

2. Copy all snapsots, which will inlcude all files in the vmfolder including all snapshots and *.vmdk

>cp -rf <localdisk>/vmfolder/ /<iscsi>/vmfolder

3. copy vmdk with vmkfstools, do I need to run this tool for all *.vmdk

>vmkfstools -i <localdisk>/vmfolder/ubuntu.vmdk <iscsi>/vmfolder/ubuntu.vmdk -d thin

does the vmkfstools -i command copy the current state of the vm?

[continuum]

you have 4 different vmdks:
name-00000*.vmdk
name-00000*-delta.vmdk
name-flat.vmdk
name.vmdk

you use cp for

name-00000*.vmdk
name-00000*-delta.vmdk

and vmkfstools -i against
name.vmdk

and you ignore:

name-flat.vmdk

you may need to adjust the CID value in name.vmdk

[idivad]

Thanks, that seemed to help!  I cloned it into a thin provisioned disk.  I have a few more question just to make sure I understand the process.

I ran the clone command on all the snapshot disk; is that ok to do?

Also what is the purpose of the other files *.vswp, it seems like I can further reduce the disk spce if I don't copy that file, what is that file used for?