Locked files not locked by any process or host - cannot remove snapshots

[Jelloir]

Long story short...

VDR had a problem with a VM.  The VDR logs intitially reported the following message:

Trouble reading files, error -3948 ( vcb api exception )

Without us realising until today VDR had been taking snapshots of the VM and never removing them so now we have a bunch of delta files for the VM and when trying to remove the shapshots we get:

Unable to access file <unspecified filename> since it is locked

Besides the basic stuff like taking and removing a manual snapshot (which did not work) I followed KB article 10051 which identified all the locked files but the owner is 00000000-00000000-0000-000000000000 but lsof produces no output on any ESX host.

So how do I unlock a file that is not locked by any process or ESX host and was most likely the result of a VDR mess up?

The output from the vmksfstools -D command on each lock file can be found in the attachment.  I notice the "RO Owner" has what appears to be an owner?

Thanks

[a_p_]

Maybe worth a try. Power down the VDR appliance and then try to delete the snapshots. If the "locked" error still occurs, restart the Management Agents on the ESXi hosts and try again.

Important: If you are running an ESX/ESXi version prior to v4.0 Update 2, the "Delete All" function might temporarily consume a huge amount of additional disk space. In this case please provide a list of files with the sizes, time stamps, ... (ls -lisa) to see how much disk space you might need. Alternatively - in case you run an older version - you could use vmkfstools on the command line clone the virtual disk including all snapshots into a new virtual disk. Therefore it is important to know the current virtual disk name mentioned in the VM's configuration (.vmx) file.

André

[Jelloir]

I turned off VDR and restarted mgmt-vmware and vmware-vpxa on both hosts, then tried removing the snapshot again from the host running the VM but it failed with the same file locked error.
When I try and remove the snapshots using a temp snapshot they are also left behind and add to the existing delta files.
Do you think I might be better trying to clone the VM?


[a_p_]

Cloning the virtual disk is certainly an option. Anyway, how many hosts do you have? I would first try to start the management agents on all hosts, to see whether a process on another host is holding the lock (e.g. still locking a file after vMotion!?).

André