vOrchestrator won't start scheduled workflow

[Ivo201110141]

Hi,

We've been running into issues where our vOrchestrator appliance will not execute workflows that are recurrently scheduled at later times. Manually executing the workflow works, even the initial run works but recurring schedules fail to start. It complains about a missing userToken and we have no idea what's wrong. This is the log for a single workflow:

2012-11-27 07:00:00.518+0100 TRACE [VSOFactoryClient] JMS Received  (edit) :  ch.dunes.model.task.Task@367dd26e[id=ff8080813a78a95d013a96d8d3cb0027,name=example,state=2] Registered : null
2012-11-27 07:00:00.518+0100 TRACE [VSOFactoryClient] JMS Received  (edit) :  ch.dunes.model.task.Task@7e824fb4[id=ff8080813a78a95d013a96d8d3cb0027,name=example,state=2] Registered : null
2012-11-27 07:00:00.519+0100 TRACE [VSOFactoryClient] Received change on an object not registered in the factory !
2012-11-27 07:00:00.518+0100 TRACE [VSOFactoryClient] Received change on an object not registered in the factory !
2012-11-27 07:00:00.533+0100 ERROR [Execution] Unable to execute a workflow, userToken is null
2012-11-27 07:00:00.533+0100 WARN  [TaskRunner] Unable to execute workflow from task
ch.dunes.util.DunesServerException: Unable to execute a workflow, userToken is null
    at ch.dunes.workflow.engine.mbean.WorkflowEngine.executeNewWorkflow(WorkflowEngine.java:508)
    at ch.dunes.workflow.engine.mbean.WorkflowEngine.executeNewWorkflowFromTask(WorkflowEngine.java:126)
    at com.vmware.o11n.service.tasks.TaskRunner.execute(TaskRunner.java:78)
    at com.vmware.o11n.service.tasks.RunnableTask.doRun(RunnableTask.java:67)
    at ch.dunes.mbean.util.scheduler.SchedulableRunnable.run(SchedulableRunnable.java:154)
    at ch.dunes.mbean.util.scheduler.ThreadPool$Worker.run(ThreadPool.java:236)
2012-11-27 07:00:00.534+0100 WARN  [RunnableTask] ==== Task 'example' failed
2012-11-27 07:00:00.542+0100 TRACE [VSOFactoryClient] JMS Received  (edit) :  ch.dunes.model.task.Task@5e33b2e7[id=ff8080813a78a95d013a96d8d3cb0027,name=example,state=-1] Registered : null
2012-11-27 07:00:00.543+0100 TRACE [VSOFactoryClient] Received change on an object not registered in the factory !
2012-11-27 07:00:00.543+0100 TRACE [VSOFactoryClient] JMS Received  (edit) :  ch.dunes.model.task.Task@14268c02[id=ff8080813a78a95d013a96d8d3cb0027,name=example,state=-1] Registered : null
2012-11-27 07:00:00.544+0100 TRACE [VSOFactoryClient] Received change on an object not registered in the factory !
2012-11-27 07:00:00.547+0100 INFO  [TaskCenter] Removing task 'example' to schedule
2012-11-27 07:00:00.563+0100 TRACE [VSOFactoryClient] JMS Received  (edit) :  ch.dunes.model.task.Task@e2ef9bd[id=ff8080813a78a95d013a96d8d3cb0027,name=example,state=1] Registered : null
2012-11-27 07:00:00.563+0100 TRACE [VSOFactoryClient] Received change on an object not registered in the factory !
2012-11-27 07:00:00.564+0100 TRACE [VSOFactoryClient] JMS Received  (edit) :  ch.dunes.model.task.Task@2e0dee91[id=ff8080813a78a95d013a96d8d3cb0027,name=example,state=1] Registered : null
2012-11-27 07:00:00.564+0100 TRACE [VSOFactoryClient] Received change on an object not registered in the factory !
2012-11-27 07:00:00.567+0100 INFO  [TaskCenter] Adding recurrent task 'example' to schedule [Wed Nov 28 07:00:00 CET 2012]

Can anyone clarify this issue for us? It would be much appreciated.

Regards,

Ivo

[cdecanini_]

There were other users reporting issues with recurrently scheduled tasks recently.

I would advise on contacting GSS to double check this is already a known issue with an upcoming fix.

Christophe.

[qc4vmware]

I have a similar problem and I was working on it prior to vacation (which I have just returned from).  By any chance are you scheduling it with once a day reoccurance?  Just before I left I had tried testing hourly and those seemed to work ok although I did not let them run for more than 24 hours to see if they started failing past that point.  It almost sounds like a token may have expired or something of that nature.  I will be doing more testing in the next week or so as I have really pushed my staff to start moving a bunch of our scheduled nightly maintenance into Orchestrator.  If this is a bug it needs to be fixed ASAP!  Hopefully it is environmental or *gasp* operator error of some sort.  I used the scheduler just fine in previous versions though.

Paul

[Matt_B1]

I am using vCO appliance v5.1.0 and I have the same issue.  I am scheduling a workflow and everything appears fine.  For that day, with the vCO java client still open, the scheduled workflows will kick off per the schedule.  At the end of the day, I close out the client and go home.  When I come back the next day, none of the scheduled jobs over night actually executed even though they show a last execution time per the schedule.  Definitely annoying...has anyone resolved this?

I found this VMware KB but I am already at v5.1.  It also highlights how one option is to the leave the client open for scheduled workflows to run.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=204173...

Here is a snippet of my logs showing the same error.

2013-05-15 06:00:00.766-0700 ERROR [Execution] Unable to execute a workflow, userToken is null

2013-05-15 06:00:00.766-0700 WARN  [TaskRunner] Unable to execute workflow from task

ch.dunes.util.DunesServerException: Unable to execute a workflow, userToken is null

  at ch.dunes.workflow.engine.mbean.WorkflowEngine.executeNewWorkflow(WorkflowEngine.java:508)

  at ch.dunes.workflow.engine.mbean.WorkflowEngine.executeNewWorkflowFromTask(WorkflowEngine.java:126)

  at com.vmware.o11n.service.tasks.TaskRunner.execute(TaskRunner.java:78)

  at com.vmware.o11n.service.tasks.RunnableTask.doRun(RunnableTask.java:67)

  at ch.dunes.mbean.util.scheduler.SchedulableRunnable.run(SchedulableRunnable.java:154)

  at ch.dunes.mbean.util.scheduler.ThreadPool$Worker.run(ThreadPool.java:236)

[iiliev]

If I recall correctly, there was a PR/SR about the same (or similar) issue, which was fixed in 5.1U1. Not sure what's the situation with 5.1.0.

[Matt_B1]

I upgraded v5.1 to v5.1.1 and the issue persists.  I also have another issue where workflows do not appear in the vSphere web client for administrators.  VMware Dev can't figure it out so I may try do a fresh v5.1.1 vCO appliance install and import the workflows back.

[iiliev]

After the upgrade to 5.1.1, does the issue persist with the workflows scheduled before you did the upgrade, or also happens with newly scheduled workflows?

[Matt_B1]

Since I am still working on another issue where the workflows do not appear in the vSphere web client, I ended up saving workflows and configs and starting over with a fresh vCO v5.1.1 appliance.  I re-created the scheduled workflows and they now execute without an issue when the Orchestrator client is closed.  I do think you had the right idea about re-creating the scheduled workflow tasks in order for it to work.

[nevrax]

For vCO Task Scheduler I have a stupid fix... until a better solution will be available:

• leave an open session of orchestrator client
• delete old schedule task and create a new one with same configuration and same workflows

In that way seems like userToken is kept as long as vCO Client is connected to server.

It's working on vCO 5.1.0 (windows setup).

If you still want to upgrade to vCenter Orchestrator 5.1.0a or 5.1.0b or 5.1.U1a (Windows Setup / Not Virtual Appliance) you can find the last versions of windows binaries inside VMware vCenter Server ISO files (VMware-VIMSetup-all-5.1.0-*.iso).

Hope this helps.

[nevrax]

Got a better solution:

• start vCenter Orchestrator Client and go to Run view
• right click and select Schedule Task as...
• search and select a workflow
  
• specify vCO credentials (user and password)
• specify  schedule details
• you can logout from vCO Client, scheduled task should work

At each scheduled run vCO will those credentials so I suppose userToken will not be null.