This morning I woke up to the VMs not working. I logged in using the Windows management client. I could not do anything with the VMs, and two had snapshots. Seldom does this happen, but when it does, I can reboot VMware, and drop any snapshots set by the nightly backup, and everything will be good. When I restarted the ESXi, two out of three were fine, but I have one VM where the entire power menu is grayed out, and it shows no snapshots. However, I have two delta VMDKs showing in the directory. I thought I would add a snapshot and then do a delete all. However, "Take Snapshot" is grayed out. I don't care about the contents of the snapshot. Any thoughts as to how I can recover the VM?
The VM icon has an exclamation point. After a long time it gave me the option to stop or retry. I said stop, and tried to take a snapshot. That failed with the error: Cannot complete the operation because file or folder already exists. [datastore-1] admin2/admin2.vmx. I now had a power on option, so I tried that. It powered up, but no snapshots show. Next I tried to make a snapshot, but it shows "Insufficient disk space on datastore." However, it shows 51 gigs of space free, which should be plenty for an empty snapshot.
When I tried to make a snapshot with the VM running, the out of disk space error was nonsensical, so I rejected that out of hand. What struck me was when I tried to make a snapshot with the VM off, I received the error: "Cannot complete the operation because file or folder already exists..." This is what I did to fix it.
1. I opened .vmx in a text editor to check which snapshot it was using. I noticed it did not contain a reference to the second snapshot. That is what I was expecting to find. It explained the "Cannot complete the operation because file or folder already exists...." error because there were 0002 snapshot files in the VM's directory.
2. As a precaution, I started the VM. With it running, I renamed the second snapshot's files by adding a z to their extension. Because it allowed me to do this, I knew for sure they were not being used by the VM.
3. I powered the VM down, and using the Client, I made a new snapshot, which I could not do before. This left me with one snapshot showing in the in the Client, but I knew that the new snapshot had to be the second.
4. From the Client I told it to delete all snapshots. It was taking a long time, which indicated to me that it was merging 0001 snapshot as well as the 0002 snapshot I had created seconds before.
5. During the posting of snapshots, I watched the diskspace diminish and could see that I was going to have a problem. The snapshot had grown huge because nobody knew it existed. I maintain a directory on each volume named reserve. They contain large files to reserve disk space for situations such as this. I deleted the directory to free up space. As I watched, the disk space got down to 30 megs, so I also powered down a non-essential VM, which returned 4 gigs of disk space. It squeaked by with 3.30 gigs to spare. Once it finished posting, I was back to 51+ gigs of free space on the volume.