VMware Cloud Community
bwiechman
Contributor
Contributor

Snapshot file not removed - or where did all my disk space go

I was testing a VM on ESX 3.5 when I had the VM crash this morning because the disk was full.

msg.hbacommon.outofspace: There is no more space for the redo log of Netxplorer-000001.vmdk...

I killed the VM, and deleted the snapshot in the snapshot manager. Now none are shown, however I'm about 60GB of space short. Digging around I found this:

# pwd

/vmfs/devices/deltadisks

# ls -lh

ls: b02c-Netxplorer-000001-delta.vmdk: No such file or directory

total 128G

-rw------- 1 root root 128G Apr 24 11:36 43031-Netxplorer-000001-delta.vmdk

-rw------- 1 root root 0 Apr 24 11:36 control

# pwd

/vmfs/volumes/esx-mtc-01:storage1/Netxplorer

# ls -lh

total 190G

-rw------- 1 root root 60G Apr 24 13:32 Netxplorer-000001-delta.vmdk

-rw------- 1 root root 256 Apr 24 08:27 Netxplorer-000001.vmdk

-rw------- 1 root root 1.5G Apr 24 08:25 Netxplorer-76c68d61.vswp

-rw------- 1 root root 128G Apr 17 16:13 Netxplorer-flat.vmdk

-rw------- 1 root root 8.5K Apr 24 08:27 Netxplorer.nvram

-rw------- 1 root root 405 Apr 17 16:13 Netxplorer.vmdk

-rw------- 1 root root 438 Apr 24 08:24 Netxplorer.vmsd

-rw-rr 1 root root 1.8K Apr 24 08:27 Netxplorer.vmx

-rw------- 1 root root 265 Apr 23 09:53 Netxplorer.vmxf

-rw-rr 1 root root 27K Mar 31 16:54 vmware-1.log

-rw-rr 1 root root 57K Apr 3 11:26 vmware-2.log

-rw-rr 1 root root 27K Apr 3 12:39 vmware-3.log

-rw-rr 1 root root 113K Apr 24 08:25 vmware-4.log

-rw-rr 1 root root 30K Apr 24 08:37 vmware.log

-r----


1 root root 9.0M Apr 24 08:24 vmware-vmx-zdump.000

Can anyone explain what the deltadisks are for?

Can I remove that 60GB *-delta.vmdk file safely? Why wouldn't it have been removed when I deleted the snapshot in the snapshot manager?

Reply
0 Kudos
5 Replies
ctfoster
Expert
Expert

The delta files are the changes created by the snapshot operation. The action of deleting the snapshot does not immediately merge the changes and null the files - it's a background task that can take some time. How long? well you've got the mother of all deltas - check out this thread

http://communities.vmware.com/message/823365

Sit tight and monitor the space.

Reply
0 Kudos
bwiechman
Contributor
Contributor

It actually grew overnight. Is there any way to verify (other than the MUI) that the snapshot is indeed disabled?

root@esx-mtc-01 Netxplorer# ls -lh

total 191G

-rw------- 1 root root 61G Apr 25 08:27 Netxplorer-000001-delta.vmdk

-rw------- 1 root root 256 Apr 24 08:27 Netxplorer-000001.vmdk

...

I don't have much room to work on this volume with that delta there, less than 1GB of free space...

Reply
0 Kudos
ctfoster
Expert
Expert

You can test from the command line using

vmware-cmd vmfs/volume/vmfslabel/VMName/VMName.vmx hassnapshot

will return

hassnapshot()=1 is the snap is active. For other commands.

Reply
0 Kudos
bwiechman
Contributor
Contributor

hassnpshot() =

So it is disabled. Basically wait and see or try the suggestions in the other thread you pointed out then I guess.

Reply
0 Kudos
bwiechman
Contributor
Contributor

In the end I discovered that I had interrupted the merge process... There is apparently no way to begin it again.

The solution that seems to have worked was to clone the VM to another datastore with available space. After some monitoring for a week or two I didn't notice any issues so I dropped the original, migrated back, and have been continuing without any issues so far.

Reply
0 Kudos