I have a few Vapps that have VM's stuck "Busy" - they have been stuck for > 4 hours.
They can not stop - the vapp can not stop... because... they are "Busy"
No operations in vCenter are waiting on these. It is like a task lost track of its self.
Does anyone have any idea how I can kick a vapp and make it stop, clear itself up... so we can start it again clean?
All the "busy" vms have the error: "This operation failed because the cell was reset."
The cell did bounce, so I assume that is what caused the issue, now trying to fix. (We have 1000+ vapps - only a few seem to have the issue)
We have had the same problem before (same error message too) and resolved the issue by poking around in the database. There is a field that gets set to RESOLVED once an operation is complete for entities such as vApp, VM etc which leaves them in a managable state. However the failure of the cell prevents the field from being updated to RESOLVED so the the entitiy remains in an unmanagble state. We found the vApp with the issue and reset the status field to RESOLVED and could then manage it via the UI.
If you can't find the field update the post and I will take a better look at what we did as I think I documented it somehwere.
Message was edited by: charliejllewe… (re-worded)
Finally we managed to solve the problem by deleting the task entry related to the hunging job in the task table. The resolved status was already there.
So to sum up on a method to solve this kind of issue, I suggest to follow these steps in the mentionned order and check after each one if the problem is solved.
1. Delete any task related to the hunging job :
delete from task where job_id='08A26E74863B416797A5D5BCFFCE13FC';
2. Make sure that the STATUS field has the value RESOLVED for the vApp on the VM_CONTAINER table and for each VM on that vApp on the VM table.
select name, creation_status from VM_CONTAINER where name = 'vApp Name'; select name, creation_status from VM where sg_id=(select sg_id from vm_container where name='vApp Name');
update VM set status='RESOLVED' where ID='326539E466934A50AEEB64BFF675EBC4';
3. Make sure for that job that the following field have these values :
- STATUS = '2'
- PROGRESS = ''
- CANCEL_REQUESTED = '0'
update jobs set status='2' where job_ID='326539E466934A50AEEB64BFF675EBC4';
4. Set the STOPTIME field to a time in the future (for example t+2min ) and wait until your reach the time you set and see if the job is done.
update jobs set stoptime='28-JUN-12 04.00.00.000000 PM' where job_ID='326539E466934A50AEEB64BFF675EBC4';
Hope this will help.