VMware Cloud Community
pops106
Enthusiast
Enthusiast
Jump to solution

Locked VMDK File?

Hello,

I have a locked VMDK file after a problem that occurred last Saturday.

During the VCB Backup somehow 5>6 VM’s got trashed.

I have restored most of them now but I have a VMDK file that I cannot attach to a VM and power it up because it says the file is locked. I also can’t copy the flat file around because it is also reporting as been locked.

The thing that worries me more is that there are deltas files associated with the VMDK file.

I have finally given in and logged it as a support request but thought it was only right to try here as well.

Thanks

Reply
0 Kudos
1 Solution

Accepted Solutions
Rob_Bohmann1
Expert
Expert
Jump to solution

This is what works for me, just had to do this today.

The command to run to list open files is

#[b]lsof |grep servername[/b] in this case the server name is LINWWD2

here is an example the processes listed in italics are the ones causing the problem. Usually, if you kill the first, you can then start your vm.

\[root@xxxxx LINWWD2]# lsof |grep LINWWD2

vmware-vm 21665 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 21665 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-mk 21668 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-mk 21668 root 14u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22255 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22255 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22256 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22256 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22259 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22259 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22260 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22260 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log[/i]

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22831 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 22832 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22833 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

So kill -9 the first process # and then list again[/b]

\[root@xxxxxx LINWWD2]# kill -9 21665

\[root@xxxxxx LINWWD2]# lsof |grep LINWWD2

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23989 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 23990 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23991 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

the ones listed now are just you listing files in the shell.

View solution in original post

Reply
0 Kudos
14 Replies
gus27
Enthusiast
Enthusiast
Jump to solution

Is there a .lock file in the directory of the VMDK file? If that's the case just try to delete it.

Guido

Reply
0 Kudos
gus27
Enthusiast
Enthusiast
Jump to solution

Sorry, I just saw that copying the flat file doesn't work either. So deleting a .lock file won't help in that case Smiley Wink

Is it possible, that the process of the trashed VM is still active? Try a

"ps ax|grep -i \[name of trashed VM]"...

Guido

conradsia
Hot Shot
Hot Shot
Jump to solution

In every case I have had a locked file I had to reboot, I would love to see what the solution is besides rebooting.

Rob_Bohmann1
Expert
Expert
Jump to solution

This is what works for me, just had to do this today.

The command to run to list open files is

#[b]lsof |grep servername[/b] in this case the server name is LINWWD2

here is an example the processes listed in italics are the ones causing the problem. Usually, if you kill the first, you can then start your vm.

\[root@xxxxx LINWWD2]# lsof |grep LINWWD2

vmware-vm 21665 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 21665 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-mk 21668 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-mk 21668 root 14u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22255 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22255 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22256 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22256 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22259 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22259 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log

vmware-vm 22260 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

vmware-vm 22260 root 3u REG 8,2 123022 652805 /home/vmware/LINWWD2/vmware.log[/i]

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22831 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 22832 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 22833 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

So kill -9 the first process # and then list again[/b]

\[root@xxxxxx LINWWD2]# kill -9 21665

\[root@xxxxxx LINWWD2]# lsof |grep LINWWD2

bash 22743 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23989 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

grep 23990 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

lsof 23991 root cwd DIR 8,2 4096 652801 /home/vmware/LINWWD2

the ones listed now are just you listing files in the shell.

Reply
0 Kudos
pops106
Enthusiast
Enthusiast
Jump to solution

Thanks everyone i will give this ago remotely later today or when im back in the office tomorrow.

I will keep my fingers crossed.

Reply
0 Kudos
pops106
Enthusiast
Enthusiast
Jump to solution

Thanks Rob, you are a star

Reply
0 Kudos
pops106
Enthusiast
Enthusiast
Jump to solution

Anyone know how to apply a delta to a VMDK.

Or get rid of it all together the disk is corrupt.

Thanks

Reply
0 Kudos
Rob_Bohmann1
Expert
Expert
Jump to solution

Ha ha , I wish I were a star... No, someone shared this with me and now I share it with you and everyone wins.... Glad to help...

Reply
0 Kudos
juchestyle
Commander
Commander
Jump to solution

Hey Rob,

Thank you for sharing. When you say server name, you are talking the host, or the vm?

Respectfully,

Kaizen!
Reply
0 Kudos
pops106
Enthusiast
Enthusiast
Jump to solution

What i did was;

browse to the /vmfs location of the VM you VMDK file you are having the problem with and then run the command with the VMname.

I had to go through 3 hosts before i found the one which was locking it.

Message was edited by:

pops106

Reply
0 Kudos
Rob_Bohmann1
Expert
Expert
Jump to solution

Thanks for reminding me Pops....

You should run this command from the host (service console) on which the vm is running on or was last running on rather. Otherwise it won't work.

The command is run against the VM lsof |grep vmname....

Reply
0 Kudos
jsteffen
Contributor
Contributor
Jump to solution

Anyone know how to do this in ESX 2.53?

Reply
0 Kudos
grasshopper
Virtuoso
Virtuoso
Jump to solution

Anyone know how to do this in ESX 2.53?

It never really worked in 2.5x. 99% of people just had to reboot the host. If you look in the vmkernel logs, you can identify the offending host that has the lock, and reboot it. Also notable is that a kill -9 of a VM can actually crash the host in rare situations, so best practice is to move all guests over to another host before attempting it.

Reply
0 Kudos
Chiel
Enthusiast
Enthusiast
Jump to solution

Can also try to Vmotion (cold or hot) to another host.

Usually i had to reboot the Host as well if i would get a lock. But last time i tried a vmotion and it released the lock.

Reply
0 Kudos