We're a cloud service provider that uses automation (via pyvmomi) to register, run, suspend, and unregister VMs. We're using vCenter 6.0, and a mix of ESXi 5.5 and 6.0 hosts.
Thousands of times a day, we programmatically go through lifecycle operations, talking to vCenter. And everything works great!
However, about once/twice per week, we have a VM that gets stuck in vCenter. We can't make power-changing operations against the VM because those operations are currently disabled. In the vCenter UI, this shows up as greyed out. When this happens, it doesn't matter how long we wait (days even). It will not un-grey out.
HOWEVER.... if we (manually) connect to the ESXi host directly, the power operations aren't disabled. We are able to poweroff/suspend the VM. And once we do that, vCenter's state magically updates to reflect this.
So two questions:
1) It seems like vCenter's state is out of sync with the ESI hosts for just the problem VM. Other VMs seem fine. What might be causing this?
2) Is there some way to programatically "kick" vCenter to force it to resync with ESXi? I realize there's constant "background" reconciliation, but when we get into this state, it doesn't seem to matter.
Thanks much!
Matt Pietrek
