Cannot remove vmware-2966.log: Device or resource busy

[bjmoore]

One of my VCBMounter commands is failing with the following info:

Error: Could not back up config file: vmname//vmware-2966.log

When I SSH into the host that the VM's on, get into its folder and try to remove the file in question, it errors out with:

rm: cannot remove `vmware-2966.log': Device or resource busy

ls -l shows I have permissions and the log file has not been modified recently:

-rw-rr 1 root root 244249 Jun 19 19:18 vmware-2966.log

lsof|grep vmware-2966.log returns nothing.

Anybody know what I can do to troubleshoot this further, other than powering down the VM and trying to delete it then?

Thanks!

Ben

[bjmoore]

Update: I have powered off the VM and tried the same steps with no luck. The file will still not delete. Bump for help.

Thanks!

Ben

[Texiwill]

Hello,

Most likely the LUN is locked. You can try using

vmkfstools -L release /vmfs/devices/disks/Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[bjmoore]

If I look in the vmkernel, I see a lock on the file by the server with the UID that ends in 0011259c7492. I've found the server, but it doesn't recognize that it has any processes or open files. ps -elf|grep vmware-2966.log and lsof|grep vmware-2966.log both return nothing.

I may vmotion the servers off of this host and reboot it to see what happens.

[kjb007]

The file handle may be open. Run vmware-cmd -l on that esx host, and see if the vm whose vmware.log file you are trying to remove is running on that host. You can vmotion the vm's off, but if the vm thinks it is writing to that log, then you still will not be able to delete that file without shutting down that vm.

-KjB

[bjmoore]

Thanks guys, I was able to do a vmkfstools --config writable, which unlocked the file and allowed me to delete it

[MaxPow]

I tried that, but it didn't work. I was able to use "fuser <filename>" to figure out which process was hanging this up, then deleting said process; worked well.

[Maarthen]

Hi,

If all else fails, you can check this procedure;

1) storage vmote the vm with the locked file to another LUN. the locked file(s) will be left behind. but the vm creates new logs on the destination lun so they can be deleted.

2) follow the vmkfstools -d procedure in the following link to find the macaddress of the esx host that's locking the file: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=10051&slic...

3) The macaddress found in the procedure above needs to be checked with all the nic macaddresses of all your esx hosts. What I did is fill a text file with the ouput of a filtered ifconfig command via plink.(plink is a tool to link your windows command prompt with a putty session)

plink root@esx101 ifconfig | findstr /i hwaddr >> macaddresses.txt

i filled the txt file with all esx servers so now i can check wich macaddress relates to wich host. (you'll need to add the hostnames yourselve in the txt file)

4) finally and this step needs to be imroved if you ask me is to put the esx host found in step 3 in maintenance mode. next is reboot the host and exit maintenance mode.

5) THE LOG FILE CAN BE DELETED!

[living8510]

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=10051&slic...

========================================================================================================

I can't open the link~~~~but I really want to know how find out the macaddress of the esx host that's locking the file?

[living8510]

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=10051&slic...

========================================================================================================

I can't open the link~~~~but I really want to know how find out the macaddress of the esx host that's locking the file?

[sridharan83]

try with this link..[.http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10051|http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10051]