How to convert thick virtual disk to thin one if it has snapshots?

[Isti]

Hi,

I have a VM on ESXi 4.1 with the first vHDD of 30GB thick provisioned.

I have quite a few snapshots that I can’t drop.

I need more space on that HDD, as such I tried cloning into a thin provisioned HDD, using:

# vmkfstools -i "Win 2003 R2.vmdk" "Win 2003 R2-thin.vmdk" -d 'thin' -a lsilogic

Destination disk format: VMFS thin-provisioned

Cloning disk 'Win 2003 R2.vmdk'...

Clone: 100% done.

When I try to remove the old/original HDD that is based on  Win 2003 R2-000008-delta.vmdk disk file, I get

Cannot remove a snapshotted disk from a virtual machine.

If I understand it correctly that’s because I have snapshots and since removing the snapshots is not an option for me, I’m looking for another way to convert the HDD from thick to thin.

Any suggestions?

PS: I tried vCenter Converter Standalone and it converts to thin, but I lose the snapshots.

[continuum]

Unregister the VM
Create new empty directory and move the vmdk and the associated flat vmdk in to it
Convert with vmkfstools -i thick.vmdk thin.vmdk -d thin
When done move the thin.vmdk into original location and adjust CID and filenames.
Then register the VM again and check if it is ok - if yes - delete the original thick vmdk

[Isti]

Thank you Ulli for your input. Since I never did something similar, I would need some more info.

Removed the Vm from the inventory.

# cp Win_2003_R2.vmdk ../tmp
# cp Win_2003_R2-flat.vmdk ../tmp
# cd ../tmp
# vmkfstools -i "Win_2003_R2.vmdk" "Win_2003_R2-thin.vmdk" -d 'thin' -a lsilogic
Destination disk format: VMFS thin-provisioned
Cloning disk 'Win 2003 R2.vmdk'...
Clone: 100% done.
# cp Win_2003_R2-thin.vmdk ../Win_2003_R2
# cp Win_2003_R2-thin-flat.vmdk ../Win_2003_R2
# ls -lh *.vmdk
...
-rw-------    1 root     root        30.0G Nov  5 22:12 Win_2003_R2-flat.vmdk
-rw-------    1 root     root        30.0G Nov 17 10:11 Win_2003_R2-thin-flat.vmdk
-rw-------    1 root     root          552 Nov 17 09:45 Win_2003_R2-thin.vmdk
-rw-------    1 root     root          524 Nov  5 22:12 Win_2003_R2.vmdk
# cat Win_2003_R2.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=2bc63d98
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfs"

# Extent description
RW 62914560 VMFS "Win_2003_R2-flat.vmdk"
...
# cat Win_2003_R2-thin.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=2bc63d98
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfs"

# Extent description
RW 62914560 VMFS "Win_2003_R2-thin-flat.vmdk"
...

Please notice that the CID remained the same.
The only -delta.vmdk that references CID=2bc63d98 is Win_2003_R2-000002.vmdk and I changed the parentFileNameHintin in this file from

# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=49d6d370
parentCID=2bc63d98
isNativeSnapshot="no"
createType="vmfsSparse"
parentFileNameHint="Win_2003_R2.vmdk"
# Extent description
RW 62914560 VMFSSPARSE "Win 2003 R2-000002-delta.vmdk"
...

to:

# cat Win_2003_R2-000002.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=49d6d370
parentCID=2bc63d98
isNativeSnapshot="no"
createType="vmfsSparse"
parentFileNameHint="Win_2003_R2-thin.vmdk"
# Extent description
RW 62914560 VMFSSPARSE "Win_2003_R2-000002-delta.vmdk"
...

Added the VM back to the inventory. Clicked Edit Settings in vSphere Client and noticed that HDD1 is still Thick.

In Win_2003_R2.vmx I have the followings:

...
scsi0:0.present = "TRUE"
scsi0:0.fileName = "Win 2003 R2-000008.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"

So, is it enough to convert/copy the base files (Win 2003 R2.vmdk and Win 2003 R2-flat.vmdk) without including the deltas?
Should I do something else?
Maybe it is a dumb question, but What's the point in copying to a different folder and not do it in place?

[a_p_]

Clicked Edit Settings in vSphere Client and noticed that HDD1 is still Thick.

Don't worry about what's shown in the GUI. A virtual disk with snapshots may always show up as "Thick" although it really isn't. I think this is because the GUI checks for the "ddb.thinProvisioned" entry in the current .vmdk file which is the snapshot file, and this does not contain the entry.

Btw. instead of editing the .vmdk file, you could have just renamed the virtual disks like:

vmkfstools -E "Win_2003_R2.vmdk" "Win_2003_R2-thick.vmdk"

vmkfstools -E "Win_2003_R2-thin.vmdk" "Win_2003_R2.vmdk"

Renaming may be less confusing in the future, since the names still match the VM's name.

André

[continuum]

> Maybe it is a dumb question, but What's the point in copying to a different folder and not do it in place?

no dumb question at all

Inside the directory exists the vmx-file, the vmsd and the snapshots.
vmkfstools notices them and then assumes it is unsafe to proceed - and in fact that assumption is justified.
By moving the vmdk to a blank directory we bypass the "is it safe ?" tests that vkfstools usually run

[Isti]

@Andre: Then how can I check to see that the VMs HDD is thin indeed? And, how can I change its size?

Good to know about renaming as I need to rename another VM. Thanks.

Thanks Ulli for the clarifications.

[a_p_]

Then how can I check to see that the VMs HDD is thin indeed?

There are several options to find out whether a base disk is thick or thin provisioned:

• delete all snapshots, then it will be displayed correctly in the GUI
• check the size in the datastore browser. For the thin provisioned virtual disk there are two size columns (size and provisioned size)
• take a look at the VM's vmware.log file

And, how can I change its size?

You can only change the size for a virtual disk which doensn't have any snapshots.

André

[Isti]

I just realized that initially I did not ask the right question as for me the goal was/is to increase the disk size of a disk with snapshots. Does it worth creating a new thread with this question, or the final answer is what Andre said? Isn't there realy any workaround?

Do you think using something like Norton Ghost could be a solution?

Whit Oracle VM, if I remember well, increasing a disk could be done by adding additional 0s to the disk file (dd if=/dev/zero... ). Would something like that work here without breaking the snapshots.

[a_p_]

I'm afraid there is no workaround, neither supported nor unsupported.

When you take a snapshot, a new delta .vmdk file is created. All subsequent writes will go to this delta file and the parent .vmdk file is accessed as read only. What's written to the delta files are data blocks with a reference to the location of the original block in the base disk. If the base disk is modified this mapping does not match anymore, and the snapshots become useless.

Please take a look at http://kb.vmware.com/kb/1015180 which explains how snapshots work in VMware products.

André

[AndySimmons]

My apologies if you're already aware of this, but VMware recommends retaining VM snapshots no longer than 72 hours, and chaining no more than 2-3 snapshots. They weren't designed to be a standalone data protection solution, and this limitation you've run into is one of many reasons they generally don't work well when used as such.

You might want to take a look at VDP. It comes bundled with all vSphere editions, so chances are you've already paid for it. There are also lots of great 3rd party products too.

[Isti]

Thanks Andy.

I was not aware of that, but I will still use this feature extensively as it is the most valuable one for me (in a test environment, with many versions, patches and unreversible test cases).

[AndySimmons]

You actually can keep using snapshots to accomplish this, but not on their own. Something like vSphere Data Protection (free with all versions of vSphere) uses snapshots while it creates a backup of the base disk, saves the image, and then commits (deletes) the snapshots from the VM, so you don't run into all the usual issues that come along with snapshots on their own. If you need to roll back to the backup, you just restore it from VDP, rather than through the snapshot menu.

You'd still have all the functionality you have with snapshots, with more flexibility, for no extra cost.

[Isti]

Thanks again Andy. I had a short view at VDP and will look more into it as it sounds promising (unfortunatelly now it is not an appropriate period for research).