cmautn1
Contributor
Contributor

Simultaneous guest operations on a single machine - The operation is not allowed in the current state.

Hello,

I am having an issue where my vRO workflows fail if I attempt to perform two guest operations on a single machine at the same time.  An example of how I can recreate the issue is start a guest script workflow that loops and attempt to use the built-in get processes from guest workflow.  Below is the error.  I am using VMTools 10.1.15 on the guest and it is running Server 2016.

I have been trying to narrow this down for some time and finally narrowed it down to two ops running at once.  Is this a limitation?  I have been researching and I have not seen anyone with this specific case.  I have not discovered anything useful in the Logs (Windows, VMware, VMTools).

Thanks!

Cody

[2017-12-21 14:32:56.313] [E] Error in (Workflow:Get processes from guest / Scriptable task (item1)#8) The operation is not allowed in the current state.

[2017-12-21 14:32:56.320] [E] Workflow execution stack:

***

item: 'Get processes from guest/item1', state: 'failed', business state: 'null', exception: 'The operation is not allowed in the current state. (Workflow:Get processes from guest / Scriptable task (item1)#8)'

workflow: 'Get processes from guest' (C98080808080808080808080808080800180808001322751030482b80adf61e7c)

0 Kudos
11 Replies
cmautn1
Contributor
Contributor

Forgot to mention,  vSphere does not create a "Task" on the VM while the script is running.  The error is something I have seen in vSphere before for failed VM tasks, so that was where I looked 1st.

0 Kudos
qc4vmware
Virtuoso
Virtuoso

Your issue sounds similar to an issue I was having.  Are you using the Guest Script Manager to run the workflows by any chance?  If so the issue is actually with how the timer is handling the sleep state.  I got an engineering build which fixed it.  If this sounds applicable let me know and I will provide you the support ticket I opened up for reference.  It took me forever to explain and get them to recreate so you will lose a day of your life if you try a fresh case.

Paul

0 Kudos
cmautn1
Contributor
Contributor

I am, yes this sounds exactly the same.  Any information I can relay would be great.

The support team you opened up with was VMWare?  I thought the Guest Script Manager was community driven.  Or is the sleep itself vRO builtin?

This is good new though, I was worried I was alone in this.

Thanks!

Cody

0 Kudos
cmautn1
Contributor
Contributor

If I modify the "Improved Sleep" and turn the entire thing into a simple sleep.  Do you think that would resolve the issue?

Turn:

pastedImage_0.png

To:

pastedImage_1.png

0 Kudos
qc4vmware
Virtuoso
Virtuoso

Yes that will work around the issue but if you have a lot of jobs running it may put undue strain on the system as I understand it.  Not best practice and why the put together the "Improved Sleep" workflow.  This is community supported but the bug is a vRO issue nothing to do with the workflows.  I put together a package that broke vRO using OOTB actions and workflows.  I never got a really great explanation of what was broken but the engineering build they gave me fixed it.  My support case number was 17569124809 for reference.

0 Kudos
cmautn1
Contributor
Contributor

Thanks so much,  I will try the work around tomorrow!  I'll let you know!

0 Kudos
zdingelit
Contributor
Contributor

Hi,

Same issue here.

Can you give me the way you fix that please !

My provisioning VM is stuck because multiple scripts are executed on the guest VM and I have the same error message.

Thank you for your reactivity !

0 Kudos
ivand
VMware Employee
VMware Employee

Hi all,

If you are on vRO 7.4, please, open a SR to VMware to getr the fix as suggested by @qc4vmware

0 Kudos
aenagy
Hot Shot
Hot Shot

cmautn1

I asked the same question here: How many simultaneous/concurrent programs/script will VMware Tools support?

In short, VMware Tools/hostd do not support more than one guest operation at a time. Someone from GSS sent me this update:

Thanks. I am rom GSS. I raised the feedback for the documentation team to clearly state "Only one vSphere API Guest OP is allowed at a time per VM. StartProgramInGuest does not wait until program finishes, so when it comes back, you can call it again to start another program"

I also asked about VMware changing the return message of "INVALID STATE" to "BUSY" in this situation and the response was:

But if you need the reporting state to be changed from "InvalidState" to "Busy", I think this should come from the product development team.

This is not a bug but simply a limitation that has not been explicitly documented. If it helps you can ask GSS to look at 18918702409.

What this means is that customers are responsible for ensuring that only one guest operation is in play per virtual machine at a time. The most logical solution is to use Orchestrator locks as a way to queue up guest operations. This helped me:

A love letter to vRO’s LockingSystem – NSFV

0 Kudos
cjanz
Contributor
Contributor

You guys are heroes - This has been driving me nuts for a long time and I had no idea it was a limitation in VMWare Tools.  I like the locking solution for guest script manager.  Unfortunately I'm using 7.5 and the issue is still here.  😞
0 Kudos
cdecanini_
VMware Employee
VMware Employee

You are right. This problem was brought to my attention and got the same information (No multiple operations on the same VM allowed at the same time) and yes using the LockingSystem was my plan to fix this but unfortunately never got the time to work on the fix.

If my answer resolved or helped you, please mark it as Correct or Helpful to award points. Thank you! Visit http://www.vcoteam.info & http://blogs.vmware.com/orchestrator for vCenter Orchestrator tips and tutorials - @vCOTeam on Twitter
0 Kudos