I have a datastore that keeps running out of disk space but I am unsure why.
No VM disks are thin provisioned. There are two VM's
1 has a 500GB disk
1 has a 100GB disk, and 2 500GB didsks.
The total is 1.6TB. The disk size is 3TB. Yet it consistently says it is out of space.
The sys_vol is on another disk and is 500GB in size and is 470GB free.
Any ideas what is going on? Is there some log file I should be looking for?
I am presently running disk consolidation on one of the VM's but I am dubious it
will help.
Thanks.
It's only the base virtual disk which is preallocated with thick provisioning. Snapshot will start with a small size, but can grow up to the provisioned size, depending on the VM's write activity.
Please take a look at the diagram at the end of https://kb.vmware.com/s/article/1015180 which explains how snapshots work.
André
If your running disk consolidation your probably had snapshots, which can be up to the same size of the disk per snapshot.
Thanks. However, why would they continue to grow if there have been no new snapshots made?
I wonder if my backup software is creating snapshots then deleting them and reclamation is not working?
Also, is it better to delete snapshots or run consolidate disks or
delete snapshot and then consolidate?
There may be snapshots even though the Snapshot Manager doesn't list any.
An easy way to find out about it, is to open the VM's settings. If the virtual disk's sizes are grayed out and/or the virtual disks' names look like "<vmname>-00000x.vmdk", then the VM has active snapshots, which you may want to consolidate.
Please note that even with thick provisioned virtual disks, consolidation - with the VM powered on - will consume some additional disk space, depending on the VM's disk write activity.
Another great way to find out whether a VM has active snapshot (along with other things) is regularly run RVTools
André
When you take a snapshot everything before it is frozen, and that doesn't change, its the delta that grows.
Why should it grow if all disk space is pre-allocated?
I can see snap A + snap B disk growth but not a continuous growth of snap B from that point.
It's only the base virtual disk which is preallocated with thick provisioning. Snapshot will start with a small size, but can grow up to the provisioned size, depending on the VM's write activity.
Please take a look at the diagram at the end of https://kb.vmware.com/s/article/1015180 which explains how snapshots work.
André
I got it. Thanks for the explanation.
If I have a backup system that makes snapshots but then deletes them,
should not the space be reclaimed by VMware? I read some things about reclaim
from CLI but is there a way to check it using either the webgui of the server or vCenter?
I have been unable to locate it.
One final question if I may. I did not receive any notification that disk space was low.
Is there a setting I am missing?
Your initial disks may be, but snapshot aren't, can't be. If you take a snapshot, delete a bunch of stuff, then fill that disk that deleted stuff needs to be somewhere since you took a snapshot before than.Its why there are recommendations to limit there long term use, like they mention in
like you mentioned I've seen backup software miss removing them so running something like the vcheck reports
vCheck vSphere - Samples - VMware {code}
or something else that can look at snapshots and orphaned snapshots is a good idea. If you have veeam one they have a tool to find the orphaned ones
If nothing goes wrong, image based backup applications delete/consolidate the snapshot they take for the backup after the job is done. However, it sometimes happens that a snapshot cannot be deleted (for whatever reason).
You can determine whether a snapshot exists with the options that I already mentioned. You can also see it from the VM's summary page, if the provisioned space is a multiple of what you've configured for the VM. Personally I use RVTools - which can even be scripted - and is free to use.
André
Okay cool.
I learned a lot from your posts.
I was unaware that snapshots grow if left more than 72hours.
I usually leave 1 whenever there is some major change to a server
just to ensure a quick wy back to a known working point without going
through a full restore.
Thanks.
Take a look at this for setting up an alarm, you can have it email you if it runs over a certain threshold.
https://uncomplicatingit.com/vmware/setting-up-custom-datastore-thresholds-in-vcenter-6-5/
It is already enabled.
Perhaps the email notification is not working.