VMware Communities > VMTN > VMware vCenter™ > VMware vCenter™ Server > Discussions

This Question is Answered

1 "correct" answer available (10 pts)
14 Replies Last post: Dec 5, 2007 2:14 PM by PotiPotato
Reply

Virtual Machine power on stalls and cannot be cancelled

May 28, 2007 8:00 AM

Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
Hello,
Actually it's all in the topic. Sometimes when I start a VM, a task starts and displays "In Progress". Everything seems good except that the tasks never finishes and cannot be cancelled. The only way I found to cancel it is to reboot the server.

As anyone already have this issue ?
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:07 AM
Click to view esiebert7625's profile Guru esiebert7625 6,790 posts since
Oct 23, 2006
Moderator
You do not have to reboot the server for that, below is a process to kill a stuck VM.

How can I kill a stuck virtual machine?
ESX 3.0 method
o Login to the service console
o You can check the VM state by typing “vmware-cmd /<path to VM directory>/server.vmx getstate”
o Type “ps -ef | grep <virtualmachinename>”
o The second column is your pid of the vmkload_app of the Virtual Machine, you can also type “ps –eaf” to see all running processes
o Type “kill -9 <pid>”
o Check VM state again, it should now be off
o Type “vmware-cmd /<path to VM directory>/server.vmx start” to power on VM

ESX 3.0 Alternate method
o Login to the service console
o Get the vmid of the VM you want to kill by typing “vm-support –x”
o Kill the VM and generate core dumps and logs by typing “vm-support –X <vmid>”
o You will be prompted if you want to include a screenshot of the VM, send an NMI to the VM and send a ABORT to the VM, you must answer Yes to the ABORT question to kill the VM. The entire process will take about 5-10 minutes to run. It will create a tar archive in the directory you run it in.

fyi...if you find this post helpful, please award points using the Helpful/Correct buttons...thanks
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:16 AM
Click to view VirtualNoitall's profile Virtuoso VirtualNoitall 1,853 posts since
Feb 17, 2006
Hello,

you can force a vm to stop by issueing the following command in the Service console :vmware-cmd <vm-cfg-path> stop Hard

Where <vm-cfg-path> is the full path to the virtual machine's config file. Remember, this is like pulling the power cord on the virtual machine but so would rebooting the host. The above should let you stop the vm without having to reboot the host.

Does the vm acutally power up and get on the network? can you ping it?

Have a look through your logs as well to see if anything stands out. Note the time of the power on and see if there are any corresponding log entries of note.
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:22 AM
in response to: esiebert7625
Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
When I use vmware-cmd :
VMControl error -11: No such virtual machine

And the VM is not in the ps, the vm-support -x doesn't list this VM ...

This one seems quit stuck !
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:27 AM
in response to: rphilippe
Click to view VirtualNoitall's profile Virtuoso VirtualNoitall 1,853 posts since
Feb 17, 2006
Hey,

Can you use vmware-cmd to try to start the virtual machine?
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:30 AM
in response to: VirtualNoitall
Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
When I try to start :
# vmware-cmd GateProtect.vmx start
VMControl error -11: No such virtual machine

When I try to check the heartbeat :
#vmware-cmd GateProtect.vmx getheartbeat
VMControl error -8: Invalid operation for virtual machine's current state
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:46 AM
in response to: rphilippe
Click to view VirtualNoitall's profile Virtuoso VirtualNoitall 1,853 posts since
Feb 17, 2006
"vmware-cmd GateProtect.vmx start "

Hello,

You need to use the full filesystem path to the virtual machine starting at /

Give it a go and let us know what happens.
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 8:51 AM
in response to: VirtualNoitall
Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
# vmware-cmd /vmfs/volumes/NetApp\ Storage/GateProtect/GateProtect.vmx start
VMControl error -11: No such virtual machine

I had tried this before but didn't work two
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 9:01 AM
in response to: rphilippe
Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
Ok ... I found a way to get the information :

# vmware-cmd /vmfs/volumes/7f8c318d-ac4ca300/GateProtect/GateProtect.vmx getstate
getstate() = off
# vmware-cmd /vmfs/volumes/7f8c318d-ac4ca300/GateProtect/GateProtect.vmx start
start() = 1

But it didn't clear out the VCenter task ...
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 9:01 AM
in response to: rphilippe
Click to view VirtualNoitall's profile Virtuoso VirtualNoitall 1,853 posts since
Feb 17, 2006
Hello,

It is still not right which is why you get the error message you are.

Issue the following command to get the list of virtual machines and their full path:
vmware-cmd -l

This will list your virtual machines and give you the full path. Then using the full path given to you in the above command try to start the virtual machine with the command mentioned in the earlier posts
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 28, 2007 9:02 AM
in response to: rphilippe
Click to view VirtualNoitall's profile Virtuoso VirtualNoitall 1,853 posts since
Feb 17, 2006
awesome. Did the virtual machine start successfully and the task is just hanging around?
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 29, 2007 1:06 AM
in response to: VirtualNoitall
Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
The task is still hanging around, the VM has not started.

This VM is only test, is there any way to turn it off kill the task and delete the VM ?
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 29, 2007 4:32 AM
in response to: rphilippe
Click to view VirtualNoitall's profile Virtuoso VirtualNoitall 1,853 posts since
Feb 17, 2006
Hello,

You can force the virtual machine off with the vmware-cmd command doing a "vmware-cmd /path/to/config/file stop hard"

Not sure if that will also clear up the job though. Can't hurt to try.

Do you see anything in the window were you kicked off the virtual machine startup?

Have you had a chance to check the logs?
Reply Re: Virtual Machine power on stalls and cannot be cancelled May 29, 2007 12:48 PM
in response to: VirtualNoitall
Click to view rphilippe's profile Novice rphilippe 14 posts since
Dec 21, 2005
No I was unable to take control over the VMs. I stopped it hard and restarted the VC service this cleared the tasks without having to reboot.

Unfortuanately I still don't really know where it came from !
Reply Re: Virtual Machine power on stalls and cannot be cancelled Dec 5, 2007 2:14 PM
Click to view PotiPotato's profile Lurker PotiPotato 1 posts since
May 15, 2007
If you are running VirtualCenter, then this problem is a symptom of your VMWare VirtualCenter Server service stopping during the start process. Check to see if this service is still running. If so, then you may need to restart it. In my experience with this issue, this will clear the "in progress" task and will allow you to start the VM again.

As I've noticed that this service stops occasionally, has anyone else had the same problem? Is there a fix for it?
Actions