Not able to delete virtual machine files from datastore in esx 3.5

[fusebox]

Hi. Hope you guys can suggest something on this. Must have seen this previously.Appreciate any help.

I had removed a VM from the inventory of the esx 3.5(update1) host,unregistered it and then went to the datastore inventor,tried deleting the files which comprised this VM. But it didnt allow me to do that.It popped up an error. I tried this in 3 methods:

1)Tried deleting it by logging into the virtual center and it gives me the error shown in fig1

2)Tried deleting it by logging into the esx host directly using vi client and gave me the error in fig2

3)Finally from the SC,tried the rm -rf

[root@esxhost1 test-vm-staging]# rm -f test-vm-staging-flat.vmdk

rm: cannot remove `test-vm-staging-flat.vmdk': Device or resource busy

#

Then,to make sure that no process was locking these files,ran a ps -eaf command to check for any stale PIDs .Found none.

Am I missing something ?

Thanks.

[khughes]

Could always try SCP'n into the vmfs volume and deleting it that way?

- Kyle

[fusebox]

Kyle,

The vmdk files I am trying to delete are in the /vmfs/volume/storage1/test-vm-staging/ directory .I didnt get what you were trying to say in your reply. I havent tried that step though. Could you let Me know what exactly you want Me to try.

Thanks.

[khughes]

Just browse to that location using your SCP program, like winSCP/fastSCP, and then just use that program do delete the files you want to remove. Almost like deleting files off of a ftp location using a ftp client

- Kyle

[espi3030]

I have had similar issues in the past, what I have found to work for me is select the directory in the lefthand pane while browsing the datastore, then delete the files of that directory in the righhand pane. Basically start from the bottom up, delete the files at the lowest level directory then delete the directory.

Highlight on the left and delete on the right. Hope this helps.

[fusebox]

Ok.This isnt working. Tried logging in via scp to the esx host as root and still getting the same error.Attaching the screenshot.

[fusebox]

The left pane and right pane workaround also didnt work. Still,the same error.

Device or Resource busy error

General Fault of file / error

Any idea why this is happening ? Any conflict/locking on the storage level?

[Troy_Clavell]

I think you have tried everything excpet for restarting the hostd process. At the service console type:

service mgmt-vmware restart

[espi3030]

See if this will help http://communities.vmware.com/message/948937 Read the information provided by rt7500, I know it's not the same exact issue but maybe it will help.

[fusebox]

Cant afford to restart the hostd pid as there some prod VMs on this esx host . Couple of them are very critical. Is there anything else you want me to try?

Appreciate all the help.

Thanks.

[Troy_Clavell]

if you are current with your patches on 3.5 restarting the hostd process will have no impact on your VM's. If you are at the base level of 3.5, you are correct, it could cause your VM's to reboot.

Maybe you can vmotion all the critical VM's off the host in question, just to be safe, then either reboot, or restart hostd. This will ensure you don't have any unexpected reboots of VM's

[fusebox]

Ok,had a look at that thread and it was kind of helpful. Went through the link posted by rt7500 . I see the below on the esx host for this VM ,who's vmdk files I am not able to delete.I was able to delete its vmlogs and vmx files though.

Observe that the pid shows as "-1" and this really shows that this vm or its files are hosed big time. Any idea how to clean up this wreck?

$ pwd

/proc/vmware/vm

$ grep -i "NYWPAPAP03-vm-staging" /

1335/names:vmid=1335 pid=-1 cfgFile="/vmfs/volumes/47e136dc-6ceb5aa0-fbf2-0017a43ba239/NYWPAPAP03-vm-staging/NYWPAPAP03-vm-staging.vmx" uuid="50 31 6d eb b5 11 bc 75-41 1c 29 b7 47 a1 58 db" displayName="NYWPAPAP03-vm-staging"

1335/userRPC: Total vmm0:NYWPAPAP03-vm-staging vmm1:NYWPAPAP03-vm-staging vmm2:NYWPAPAP03-vm-staging vmm3:NYWPAPAP03-vm-staging

1336/userRPC: Total vmm0:NYWPAPAP03-vm-staging vmm1:NYWPAPAP03-vm-staging vmm2:NYWPAPAP03-vm-staging vmm3:NYWPAPAP03-vm-staging

1337/userRPC: Total vmm0:NYWPAPAP03-vm-staging vmm1:NYWPAPAP03-vm-staging vmm2:NYWPAPAP03-vm-staging vmm3:NYWPAPAP03-vm-staging

1338/userRPC: Total vmm0:NYWPAPAP03-vm-staging vmm1:NYWPAPAP03-vm-staging vmm2:NYWPAPAP03-vm-staging vmm3:NYWPAPAP03-vm-staging

Hope this output helps in identifying the problem.

[fusebox]

If nothing works out,then I will have to restart the hostd process on this esx host. And,yes this host is not up2date .Its running on the base esx 3.5. This box is actually being used as a poc since last 2 months. Do you think applying the update 1 of esx 3.5 will fix this? Can you take a look at the latest output I posted and see if there is any other way to clean it up other than this hostd restart? If no, then I will have no other choice left.

Thanks for your time.

[fusebox]

Ok,tried this step too just now.Restarted the hostd process and even then still not able to delete that file.

[fusebox]

Answered