pops106
Enthusiast
Enthusiast

Locked VMDK File?

Jump to solution

Hello,

I have a locked VMDK file after a problem that occurred last Saturday.

During the VCB Backup somehow 5>6 VM’s got trashed.

I have restored most of them now but I have a VMDK file that I cannot attach to a VM and power it up because it says the file is locked. I also can’t copy the flat file around because it is also reporting as been locked.

The thing that worries me more is that there are deltas files associated with the VMDK file.

I have finally given in and logged it as a support request but thought it was only right to try here as well.

Thanks

0 Kudos
1 Solution

Accepted Solutions
Rob_Bohmann1
Expert
Expert

This is what works for me, just had to do this today.

The command to run to list open files is

#[b]lsof |grep servername[/b] in this case the server name is LINWWD2

here is an example the processes listed in italics are the ones causing the problem. Usually, if you kill the first, you can then start your vm.

\[root@xxxxx LINWWD2]# lsof |grep LINWWD2

vmware-vm 21665 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 21665 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-mk 21668 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-mk 21668 root 14u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22255 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22255 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22256 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22256 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22259 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22259 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22260 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22260 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log[/i]

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22831 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 22832 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22833 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

So kill -9 the first process # and then list again[/b]

\[root@xxxxxx LINWWD2]# kill -9 21665

\[root@xxxxxx LINWWD2]# lsof |grep LINWWD2

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23989 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 23990 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23991 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

the ones listed now are just you listing files in the shell.

View solution in original post

0 Kudos
14 Replies
gus27
Enthusiast
Enthusiast

Is there a .lock file in the directory of the VMDK file? If that's the case just try to delete it.

Guido

0 Kudos
gus27
Enthusiast
Enthusiast

Sorry, I just saw that copying the flat file doesn't work either. So deleting a .lock file won't help in that case Smiley Wink

Is it possible, that the process of the trashed VM is still active? Try a

"ps ax|grep -i \[name of trashed VM]"...

Guido

conradsia
Hot Shot
Hot Shot

In every case I have had a locked file I had to reboot, I would love to see what the solution is besides rebooting.

Rob_Bohmann1
Expert
Expert

This is what works for me, just had to do this today.

The command to run to list open files is

#[b]lsof |grep servername[/b] in this case the server name is LINWWD2

here is an example the processes listed in italics are the ones causing the problem. Usually, if you kill the first, you can then start your vm.

\[root@xxxxx LINWWD2]# lsof |grep LINWWD2

vmware-vm 21665 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 21665 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-mk 21668 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-mk 21668 root 14u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22255 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22255 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22256 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22256 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22259 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22259 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22260 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22260 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log[/i]

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22831 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 22832 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22833 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

So kill -9 the first process # and then list again[/b]

\[root@xxxxxx LINWWD2]# kill -9 21665

\[root@xxxxxx LINWWD2]# lsof |grep LINWWD2

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23989 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 23990 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23991 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

the ones listed now are just you listing files in the shell.

View solution in original post

0 Kudos
pops106
Enthusiast
Enthusiast

Thanks everyone i will give this ago remotely later today or when im back in the office tomorrow.

I will keep my fingers crossed.

0 Kudos
pops106
Enthusiast
Enthusiast

Thanks Rob, you are a star

0 Kudos
pops106
Enthusiast
Enthusiast

Anyone know how to apply a delta to a VMDK.

Or get rid of it all together the disk is corrupt.

Thanks

0 Kudos
Rob_Bohmann1
Expert
Expert

Ha ha , I wish I were a star... No, someone shared this with me and now I share it with you and everyone wins.... Glad to help...

0 Kudos
juchestyle
Commander
Commander

Hey Rob,

Thank you for sharing. When you say server name, you are talking the host, or the vm?

Respectfully,

Kaizen!
0 Kudos
pops106
Enthusiast
Enthusiast

What i did was;

browse to the /vmfs location of the VM you VMDK file you are having the problem with and then run the command with the VMname.

I had to go through 3 hosts before i found the one which was locking it.

Message was edited by:

pops106

0 Kudos
Rob_Bohmann1
Expert
Expert

Thanks for reminding me Pops....

You should run this command from the host (service console) on which the vm is running on or was last running on rather. Otherwise it won't work.

The command is run against the VM lsof |grep vmname....

0 Kudos
jsteffen
Contributor
Contributor

Anyone know how to do this in ESX 2.53?

0 Kudos
grasshopper
Virtuoso
Virtuoso

Anyone know how to do this in ESX 2.53?

It never really worked in 2.5x. 99% of people just had to reboot the host. If you look in the vmkernel logs, you can identify the offending host that has the lock, and reboot it. Also notable is that a kill -9 of a VM can actually crash the host in rare situations, so best practice is to move all guests over to another host before attempting it.

0 Kudos
Chiel
Enthusiast
Enthusiast

Can also try to Vmotion (cold or hot) to another host.

Usually i had to reboot the Host as well if i would get a lock. But last time i tried a vmotion and it released the lock.

0 Kudos