Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)

[mattad]

I have a running Ubuntu guestOS installation on a Win7 hostOS.

The corresponding *.vmdk file has a size of approx 8.5 GB (!).

When I start the VM and enter in a terminal:

df -h

then I can see that the partition contains only 3.9 GB  (="used").

So there should be a lot of space which can be compacted/shrinked.

I exit the guestOS and go to menu:

Vmware machine settings->hardware->Harddisc->Utilities->Compact

and started a "compact" operation.

Much to my surprise the *.vmdk file is afterwards (almost) as big as before.

So compacting seems NOT to work.

Why?

A defragmentation does not help.

Some with VmWare Player.

And same with cmdline command

vmware-vdiskmanager.exe -k Ubuntu.vmdk

How else can I compact shrink the *.vmdk?

Matt

[WoodyZ]

Read Shrink a Virtual Disk in the Help File!

[mattad]

Ok, I did it like the Help file told:

However in this case the tab  "Shrink" in "vmware-toolbox" (executed as root) is EMPTY.

So I cannot select a partition to shrink.

Obviously at least ONE partition must exist.

So I guess there is a bug in vmware-toolbox

Matt

[WoodyZ · Accepted solution]

Assuming the VMware virtual disk is not pre-allocated and the Filesystem is ext4 you will have to manually prepare it to be shrunk.

In the VM, in a Terminal become root and then copy and paste the following command, as is, and then press Enter:

dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile

Wait for the command prompt to return and then shutdown the VM and then from the Virtual Machine Settings for the Hard Disk select the Compact command from the Utilities button.

[mattad]

Wow!! It works

From where did you get this trick?

Why is this nowhere documented?

Is this only for ext4 file systems or for ext3 as well?

"Compact" operation should do that automatically!

Matt

[WoodyZ]

From where did you get this trick?

The dd command has been around since the early days of Unix and I've been using it for one thing or another for so long, long before I used it in a VM, that I don't remember.

Why is this nowhere documented?

The dd command is documented in the man pages.  In a Terminal type: man dd

Is this only for ext4 file systems or for ext3 as well?

It can work at the device level or filesystem level and at the filesystem level it should be able to work on any mounted filesystem when run under an OS that supports dd.  You should be able to use VMware Tools Shrink feature on ext3 filesystem.

"Compact" operation should do that automatically!

Would be nice but I don't see it happening anytime soon.

[wdeviers]

I think what he was asking was "why did it work?"

The point is that dd'ing zeros over a portion of a disk is going to make it look unused to the code that does vmdk compaction.  Any way of writing zeros to a vmdk should work, there's no magic in dd.  Lets say you're using LVM and are done with a volume, you could do something like...

dd if=/dev/zero of=/dev/vg0/OldLVGoneNow

-... wait....wait...wait...

lvremove OldLVGoneNow

the run the compaction and it should reclaim the space.  You could just as easily do

cat /dev/zero > /dev/vg0/OldLVGoneNow

So what he had you do was create a file that was all zeros and completely filled the partition, then remove the file.  It made sure that all freespace on the partition was filled with zero, so that the compactor could reap the space afterward.

[WoodyZ]

I think what he was asking was "why did it work?"

I disagree with that as he asked 3 explicit and specific questions.

From where did you get this trick?
Why is this nowhere documented?
Is this only for ext4 file systems or for ext3 as well?

Which I answered and if he was asking what you think, he could have replied again asking that question.

As to dd or cat, I use both for this at times and obviously either can be used.  Using Shrink from the VMware Tools UI zeros out the free space before compacting and the fact that dd and or cat is using /dev/zero it's obvious it's writing a file full of zeros and that enables the disk to be rewritten during the compact stage and not have to rewrite all the zeros when creating the new disk.

[wdeviers]

In general, somebody who's using Linux but doesn't know what 'dd' does was probably under the impression that dd itself was creating space for compaction.  I thought it was worthwhile to seperate the magic of 'dd' from the magic of zeros for future people who were searching for this.  A friend of mine found this post and asked me to clarify (in person), so I thought a general clarifcation would be valuable.

Best,

Wes

[nico76]

Hi, I've the same problem but the OS Guest is Windows 7 Pro.

Can you help me?

Thnaks

[WoodyZ]

nico76 wrote: Hi, I've the same problem but the OS Guest is Windows 7 Pro.

Have  look at: Configuring and Maintaining Virtual Hard Disks

If Compact does not do a thorough job then you might have to use VMware Tools.

In an Administrator Command Prompt:

cd "C:\Program Files\VMware\VMware Tools"

C:\Program Files\VMware\VMware Tools> VMwareToolboxCmd help disk

If you have not installed VMware Tools within the Windows Guest OS then you'll not have the VMwareToolboxCmd available.

Maybe some pictures will help!

(Click image to enlarge.)

[nico76]

thanks

[Mario88]

WoodyZ schrieb:

Assuming the VMware virtual disk is not pre-allocated and the Filesystem is ext4 you will have to manually prepare it to be shrunk.

In the VM, in a Terminal become root and then copy and paste the following command, as is, and then press Enter:

dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile

Wait for the command prompt to return and then shutdown the VM and then from the Virtual Machine Settings for the Hard Disk select the Compact command from the Utilities button.

Additional remark:

You can further reduce the size of the virtual disk file if you do defragmentation in the guest os.

For ext4, just execute

$ sudo e4defrag /dev/sdaX

(replace sdaX with the name of your partition. If you don't know the name, see How do I view all available HDD's partitions? - Ask Ubuntu)

Then, execute the command from WoodyZ and compact your disk. Done.

My results are as follows:

Used disk space in guest: 3GB

Virtual disk file size (original): 6.5GB

Virtual disk file size after zeroing free space only: 4.9GB

Virtual disk file size after defragmentation and zeroing free space: 3.3GB

[Supraba]

What to do to expand disk space allocated??It ran out of space while cloning git.

[WoodyZ]

Supraba wrote: What to do to expand disk space allocated??It ran out of space while cloning git.

1. You've posted in to thread that is the exact opposite of your issue!

2. Since you've not provided any information about what VMware product and version or Host OS and or Guest OS versions you're using then I'd suggest you read the product documentation and search the VMware KB and or forums and or Google as this has already been covered ad nauseum.

[John39]

@WoodyZ

dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile

Thank you for this - I can confirm it also works with the latest Ubuntu 20.04 and the internal VMware Workstation tool is still broken as of version 16