VMware Communities
bartgrefte
Contributor
Contributor

Home Assistant, disk size is not dynamically shrinking

A while ago I started using Home Assistant in a virtual computer using VMWare Workstation 15.5.6. This works fine, except for one thing.

The VMDK keeps growing and growing, while the actual storage usage inside the VM doesn't grow that much. Currently the storage usage inside the VM is 6.24GB, while the VMDK on the host (W10) has grown to 12GB.

When I opened  a github-issue about this, I was told that Home Assistant has been equipped with fstrim that's supposed to solve this problem, and it does on KVM/Proxmox and other virtualization environments.

Just not in VMWare Workstation.

I have confirmed fstrim is running, also manually ran it, but no decrease in size of the VMDK, nor does running defragment followed by compact help in the VM menu.

Does anyone happen to know anything about this?

Reply
0 Kudos
8 Replies
bluefirestorm
Champion
Champion

For the compact to work, the guest VM needs to punch zeroes into the guest OS filesystem to create contiguous zeroes.

For a Linux based system, this can be done using dd into a file with /dev/zero and then delete that file. But I suppose the expectation was that the fstrim would not require the need to do that.

One other possibility is if there is encryption of the VM disk. As explained above the punching of zeroes would be required for compression to work, but if the VM OS is encrypting the contents, guess what those zeroes being punched in won't be zeroes. When the guest OS is encrypting its disk contents, it is best to use a preallocated virtual disk storage otherwise you would have this runaway growth of the virtual disk that cannot be compacted.

Reply
0 Kudos
wila
Immortal
Immortal

Hi,

Sadly vmware-tools does not work yet with fstrim.
There's a github issue open for that and yes it would be very nice if that works as it would prevent the VM blowing up to use full space before the actual reclaiming process.

You still have to manually reclaim space now.

If you want to prevent the "size blow up" issue then you would have to use vmware-vdiskmanager -k with the VM shut down.

All steps are described here:

https://wiki.vi-toolkit.com/index.php?title=Shrink_guest_on_hosted_platform

--
Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
bartgrefte
Contributor
Contributor

@bluefirestorm 

As far as I know there's no encryption involved.

Are you 100% sure that the dd method will work in combination with Home Assistant? Since I do not want to end up with an even bigger VMDK.

 

"use vmware-vdiskmanager -k with the VM shut down."

Forgot to mention, already tried that. It indicated it was "shrinking" the VMDK, but it did not actually shrink, it remained the same size...

Reply
0 Kudos
wila
Immortal
Immortal


@bartgrefte wrote:
 

"use vmware-vdiskmanager -k with the VM shut down."

Forgot to mention, already tried that. It indicated it was "shrinking" the VMDK, but it did not actually shrink, it remained the same size...


Then you'll have to use any of the other methods I describe in the article that I linked.
Note that it will first increase the disk to full possible size before it will shrink, so make sure you have plenty of free space.

--
Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
bartgrefte
Contributor
Contributor

"Note that it will first increase the disk to full possible size "

And that happens every single time? The VMDK is stored on a SSD, don't know the amount of writes it's supposed to be able to handle but if this is gonna be a regular thing....

Plus, isn't running the zero fill commands doing the same as fstrim is doing automatically? Since fstrim has no effect at all.

Reply
0 Kudos
wila
Immortal
Immortal

Hi,


@bartgrefte wrote:

And that happens every single time? The VMDK is stored on a SSD, don't know the amount of writes it's supposed to be able to handle but if this is gonna be a regular thing....

That scary "writes are limited on an SSD" story isn't really relevant anymore on today's SSDs, if it ever was. I'm suggesting to run this for the FIRST reclaim.


Plus, isn't running the zero fill commands doing the same as fstrim is doing automatically? Since fstrim has no effect at all.


No, it is not the same thing. Zero fill will write out zero's to every sector on the file system that is not currently used by your files. It does this regardless of what the virtual disk supports.

Whereas fstrim only works for virtual disks that support the discard command.

Also from what I understand fstrim will discard only sectors that have been declared deleted since the last fstrim run.

--
Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
bartgrefte
Contributor
Contributor

Late update, I ran cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill , followed by vmware-vdiskmanager -k "C:\Users\Bart\Documents\Virtual Machines\Home Assistant OS 64-bit\haos_ova-6.2.vmdk" after I shutdown the VM. That seems to work, the VMDK size went from 13.5GB to 6.9GB 🙂

Now to figure out why HASS does not seem to be able to do this by itself, according to someone who replied on the Github report HASS can do this on other virtualization platforms, so what's different about VMware?

Reply
0 Kudos
bluefirestorm
Champion
Champion

Perhaps one consideration is to reinstall Home Assistant to a preallocated virtual disk. If re-installation is not desirable, you could try cloning the existing virtual disk to one that is pre-allocated to a fixed size (for the sake of example: 10GB) that would meet the HAOS VM usage requirements .

Otherwise, it is quite likely that regular/periodical maintenance (punching zeroes and then compacting) of the virtual disk would be required.

Reply
0 Kudos