14 Replies Latest reply on Jan 1, 2015 10:30 AM by WoodyZ

    Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)

    mattad Hot Shot

      I have a running Ubuntu guestOS installation on a Win7 hostOS.

      The corresponding *.vmdk file has a size of approx 8.5 GB (!).

       

      When I start the VM and enter in a terminal:

       

      df -h

       

      then I can see that the partition contains only 3.9 GB  (="used").

       

      So there should be a lot of space which can be compacted/shrinked.

       

      I exit the guestOS and go to menu:

       

      Vmware machine settings->hardware->Harddisc->Utilities->Compact

       

      and started a "compact" operation.

       

      Much to my surprise the *.vmdk file is afterwards (almost) as big as before.

      So compacting seems NOT to work.

       

      Why?

       

      A defragmentation does not help.

      Some with VmWare Player.

      And same with cmdline command

       

      vmware-vdiskmanager.exe -k Ubuntu.vmdk

       

      How else can I compact shrink the *.vmdk?

       

      Matt

        • 1. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
          WoodyZ Guru

          Read Shrink a Virtual Disk in the Help File!

          • 2. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
            mattad Hot Shot

            Ok, I did it like the Help file told:

             

            However in this case the tab  "Shrink" in "vmware-toolbox" (executed as root) is EMPTY.

            So I cannot select a partition to shrink.

             

            Obviously at least ONE partition must exist.

             

            So I guess there is a bug in vmware-toolbox

             

            Matt

            • 3. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
              WoodyZ Guru

              Assuming the VMware virtual disk is not pre-allocated and the Filesystem is ext4 you will have to manually prepare it to be shrunk.

               

              In the VM, in a Terminal become root and then copy and paste the following command, as is, and then press Enter:

               

              dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile

               

              Wait for the command prompt to return and then shutdown the VM and then from the Virtual Machine Settings for the Hard Disk select the Compact command from the Utilities button.

              • 4. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                mattad Hot Shot

                Wow!! It works

                 

                From where did you get this trick?

                 

                Why is this nowhere documented?

                 

                Is this only for ext4 file systems or for ext3 as well?

                 

                "Compact" operation should do that automatically!

                 

                Matt

                • 5. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                  WoodyZ Guru
                  From where did you get this trick?

                   

                  The dd command has been around since the early days of Unix and I've been using it for one thing or another for so long, long before I used it in a VM, that I don't remember.

                   

                  Why is this nowhere documented?

                   

                  The dd command is documented in the man pages.  In a Terminal type: man dd

                   

                  Is this only for ext4 file systems or for ext3 as well?

                   

                  It can work at the device level or filesystem level and at the filesystem level it should be able to work on any mounted filesystem when run under an OS that supports dd.  You should be able to use VMware Tools Shrink feature on ext3 filesystem.

                   

                  "Compact" operation should do that automatically!

                   

                  Would be nice but I don't see it happening anytime soon.

                  • 6. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                    wdeviers Lurker

                    I think what he was asking was "why did it work?"

                     

                    The point is that dd'ing zeros over a portion of a disk is going to make it look unused to the code that does vmdk compaction.  Any way of writing zeros to a vmdk should work, there's no magic in dd.  Lets say you're using LVM and are done with a volume, you could do something like...

                     

                    dd if=/dev/zero of=/dev/vg0/OldLVGoneNow

                    -... wait....wait...wait...

                    lvremove OldLVGoneNow

                     

                     

                    the run the compaction and it should reclaim the space.  You could just as easily do

                    cat /dev/zero > /dev/vg0/OldLVGoneNow

                     

                    So what he had you do was create a file that was all zeros and completely filled the partition, then remove the file.  It made sure that all freespace on the partition was filled with zero, so that the compactor could reap the space afterward.

                    • 7. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                      WoodyZ Guru

                      I think what he was asking was "why did it work?"

                       

                      I disagree with that as he asked 3 explicit and specific questions.

                      From where did you get this trick?
                      Why is this nowhere documented?
                      Is this only for ext4 file systems or for ext3 as well?

                       

                      Which I answered and if he was asking what you think, he could have replied again asking that question.

                       

                      As to dd or cat, I use both for this at times and obviously either can be used.  Using Shrink from the VMware Tools UI zeros out the free space before compacting and the fact that dd and or cat is using /dev/zero it's obvious it's writing a file full of zeros and that enables the disk to be rewritten during the compact stage and not have to rewrite all the zeros when creating the new disk.

                      • 8. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                        wdeviers Lurker

                        In general, somebody who's using Linux but doesn't know what 'dd' does was probably under the impression that dd itself was creating space for compaction.  I thought it was worthwhile to seperate the magic of 'dd' from the magic of zeros for future people who were searching for this.  A friend of mine found this post and asked me to clarify (in person), so I thought a general clarifcation would be valuable.

                         

                        Best,

                         

                        Wes

                        • 9. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                          nico76 Lurker

                          Hi, I've the same problem but the OS Guest is Windows 7 Pro.

                          Can you help me?

                          Thnaks

                          • 10. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                            WoodyZ Guru

                            nico76 wrote: Hi, I've the same problem but the OS Guest is Windows 7 Pro.

                             

                            Have  look at: Configuring and Maintaining Virtual Hard Disks

                             

                            If Compact does not do a thorough job then you might have to use VMware Tools.

                             

                            In an Administrator Command Prompt:


                            cd "C:\Program Files\VMware\VMware Tools"

                            C:\Program Files\VMware\VMware Tools> VMwareToolboxCmd help disk


                            If you have not installed VMware Tools within the Windows Guest OS then you'll not have the VMwareToolboxCmd available.

                             

                            Maybe some pictures will help!

                             

                            (Click image to enlarge.)

                            VMware_Tools_VMwareToolboxCmd.png

                             

                            Admin_Command_Prompt_VMware_Tools_VMwareToolboxCmd.png

                            • 12. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                              Mario88 Novice

                              WoodyZ schrieb:

                               

                              Assuming the VMware virtual disk is not pre-allocated and the Filesystem is ext4 you will have to manually prepare it to be shrunk.

                               

                              In the VM, in a Terminal become root and then copy and paste the following command, as is, and then press Enter:

                               

                              dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile
                              Wait for the command prompt to return and then shutdown the VM and then from the Virtual Machine Settings for the Hard Disk select the Compact command from the Utilities button.

                               

                               

                              Additional remark:

                              You can further reduce the size of the virtual disk file if you do defragmentation in the guest os.

                               

                              For ext4, just execute

                              $ sudo e4defrag /dev/sdaX

                              (replace sdaX with the name of your partition. If you don't know the name, see How do I view all available HDD's partitions? - Ask Ubuntu)

                               

                              Then, execute the command from WoodyZ and compact your disk. Done.

                               

                              My results are as follows:

                              Used disk space in guest: 3GB

                              Virtual disk file size (original): 6.5GB

                              Virtual disk file size after zeroing free space only: 4.9GB

                              Virtual disk file size after defragmentation and zeroing free space: 3.3GB

                              • 13. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                                Supraba Lurker

                                What to do to expand disk space allocated??It ran out of space while cloning git.

                                • 14. Re: Compact/shrink *.vmdk does NOT work (for Ubuntu guestOS)
                                  WoodyZ Guru

                                  Supraba wrote: What to do to expand disk space allocated??It ran out of space while cloning git.

                                   

                                  1. You've posted in to thread that is the exact opposite of your issue!

                                   

                                  2. Since you've not provided any information about what VMware product and version or Host OS and or Guest OS versions you're using then I'd suggest you read the product documentation and search the VMware KB and or forums and or Google as this has already been covered ad nauseum.