vRA 6.2 - error while loading a ASD workflow user presentation

[Czernobog]

Hi,

I have a vRO workflow which I have integrated in vRA as a ASD workflow. The workflow stops at an user interaction. Some of the values in the presentation field are predefined values, that are generated after the workflow starts and loadedi nto the presentation.

When I try to answer the user interaction, the vRA GUI starts to load the fields, but shortly after starting I get the error:

The vRO workflow validation shows no errors, I can run it just fine with the vRO client. When I completely remove the predefined values, the presentation GUI loads the Summary and description fields of the Manual User Interaction, however this end with an error as well after a short time.

2015-12-11 13:40:39,436 vcac: [component="cafe:shell" priority="ERROR" thread="tomcat-http--1278" tenant=""] com.vmware.vcac.shell.ErrorManagerImpl.logToApacheCommons:59 - <92df07b1> Unexpected excepti

on was caught

com.vmware.vcac.platform.framework.error.ErrorCodeException: 504

        at Unknown.gk(Unknown Source)

        at Unknown.pk(Unknown Source)

        at Unknown.Xpc(Unknown Source)

        at Unknown.v2h(Unknown Source)

        at Unknown.LCe(Unknown Source)

        at Unknown.JCe(Unknown Source)

        at Unknown.jzi(Unknown Source)

        at Unknown.mzi(Unknown Source)

        at Unknown.dBi(Unknown Source)

        at Unknown.hBi(Unknown Source)

        at Unknown.aBi/<(Unknown Source)

        at Unknown.p(Unknown Source)

        at Unknown.gadgets.util.makeClosure/<(Unknown Source)

        at Unknown.anonymous(Unknown Source)

Any ideas? Does it have to do with a timeout when loading the presentation?

[bdamian]

Are you using vRO Actions to predefine values? And if you are doing this, are those calls passing some value from another field?

In any case, please remove all "predefined" values and tell me if the problem persists.

D.

[Czernobog]

I've removed all predefined value properties from the user interaction in vRO. Same error.

On a sidenote - the main workflow presentation itself does not accept any inputs. The reason is, I have to generate some lists on basis of the __asd_requestedBy parameter, which does not get passed to vRO, until the workflow is started in vRA. Adfter a user requests the workflow, a few arrays are created in vRO and the resultst are supposed to be pushed as predefined values in the user interaction.

[bdamian]

You can use __asd_requestedBy to create lists before submit the form. But this is another discussion. First of all you need to solve the error. Can you show some screenshots?

D.

[Czernobog]

Screenshots of what? The workflow?

[bdamian]

Have you any ASD blueprint that actually works? If yes, take the workflow of the ASD that doesn't work and clear all additional code you've added on "Presentation" tab and try again. Then you need to refresh the ADS request form with the las icon in this screenshot:

If the error persist, please let me know all input parameters type.

If no ADS blueprint works, please try a new one using a Workflow with no input parameters.

D.

[Czernobog]

When I remove all predefined values from the input form, the ASD form is loading, however only after I have increase the global time out value on the vRA, like here: http://kb.vmware.com/kb/2127518

I have to increase it to 240000ms (4 mins) to load the form.

Still, loading the form with predefined values does not work, even after increasing the timeout value to 360000. There should be 2 fields which are filled with predefined values, like this:

Array of VCACCAFE:CatalogResource:

     

Array of VCACCAFE:BusinessGroup

Both are arrays of values that are calculated before the user interaction form is loaded. I've tried to increase the timeout to 600000 ms (10 min) and still get the error. Even if it did work though, it would be unacceptable because of the long loading time.

[bdamian]

It seems to be an unusual WF. It's difficult to understand it without seeing it. If you upload a simplify version of the WF, I can test it in my lab.

D.

[SeanKohler]

Okay... I spent some time thinking about whether or not to respond to this one.  I think in the context of community, I just should.  I think a lot of people are following a similar path as you with regard to form design, and there are (currently) MUCH faster ways to gain lists of objects into a field for user presentation. Also, Business Groups (unless you are at 6.2.2+ of the plugin) load very slowly and have limitations in the vRO/vRA plugin. So look into that by checking your version and seeing how long it takes your BG list to load in the vRO inventory page. (upgrade your plugin for some improvement)

I don't know what your error is.  Like bdamian, I would need to see more of your workflow and determine if you are using any External reference actions up in ASD  (or if it is all vRO presentation).  But I can tell from your two smaller screenshots that you are pulling an Array of two of the SLOWEST object types into your form.  (each of which loads slower the more items are created)  This slowness likely presents many timeouts and likely is the primary cause of your error.  (though I have seen ASD forms get into a bad state... and sometimes it is just better to delete them and point back at the workflow to start over)

Regardless of error and extended timeout, you stated that the load time (and change time) is long and not a usable user experience.  I find if you solve that problem, other problems tend to go away.

Carefully consider these screenshots, the programming practice it represents (objects-lookup via action, string-presentation, string-to-object-convert), and the blazing speed for what it is doing (50+ business groups, 40+ IaaS Blueprints).

Then in workflow execution (post request)... run actions to lookup the object selected (as a string) by the user so you can use the object.  I got the string from the ProvisioningGroup list... but look up the BusinessGroup object.

The VCACCAFE side of the plugin has a search filter implemented for Server.findAllForType(VCACCAFE:BusinessGroup, "filtername") or you can use the EntitiesFinder if you have the ID.  These options are pretty fast, but the time doesn't matter as much in workflow execution as compared to user presentation.

Unless we see a major object topology change in vRO, I do not foresee an object based array in a form being faster than a string array. (Prove me wrong VMware.) Especially in that vRO action execution can filter the object results to a smaller list of strings based on various criteria.  Complex objects as form inputs are JUST TOO DAMN SLOW. And though using the vRA ASD External mechanism to call vRO Actions that filter/limit the amount of objects returned can make it faster than pulling the full object list, we can do the same as a list of strings which will STILL likely be faster.  (test and verify always)

I hope something in all that helps you.  Best of luck in your automation efforts!

[SeanKohler]

With regard to business group timing (in the VRO plugin, as REST is fast-just to be clear)...  here is some information from one of my support requests earlier this year...

It clearly shows how SLOW working with Business Groups are.  That translates into SLOW in your forms.  It was improved after this... as this was the 6.1 plugin, but it still isn't great.

Additional Info

[2015-04-23 11:24:44.279] [I] Cafe Lookup Start: 1429813470613
[2015-04-23 11:24:44.279] [I] Cafe Lookup End: 1429813484278
[2015-04-23 11:24:44.279] [I] Total time: 13.665 seconds
[2015-04-23 11:24:44.279] [I] Total BGs: 8

[2015-04-23 11:24:44.312] [I] IaaS Lookup Start: 1429813484280
[2015-04-23 11:24:44.312] [I] IaaS Lookup End: 1429813484312
[2015-04-23 11:24:44.313] [I] Total time: 0.032 seconds
[2015-04-23 11:24:44.313] [I] Total PGs: 8

Edit:

And to see how fast the times ramp upward with additional business groups...

Run Times: ~22 seconds to grab 10 business groups with vCACCAFEEntitiesFinder.getBusinessGroups(cafeHost) 

[2015-05-29 10:24:18.391] [I] Cafe Lookup Start: 1432920235813

[2015-05-29 10:24:18.391] [I] Cafe Lookup End: 1432920258390

[2015-05-29 10:24:18.391] [I] Total time: 22.577 seconds 

[2015-05-29 10:24:18.391] [I] Total BGs: 10

[Czernobog]

Thanks you very much for this really informative answer!

I will definitely look into using the information to optimize the workflow run speed. What you said about using string values vs. more complex values makes total sense, this was a tip given to me by a member of professional services a while ago and I admin I forgot about it. That would also allow for the form to be cleaner, since all of the additional attributs of the VMs and BGs will not be displayed...

Attached you will find the part of my workflow in question. I have cut the content coming after the user interaction, because it is irrelevant, so there will be validation warnings regarding unused attributes.

To get it to run, you have to fill the attribute "vCACCAFEHosts" in the internal "Get VMs and BGs of WF User".

A bit of context - the completed workflow will allow a vRA User to create NSX DFW firewall rules and I need the input validation in the user interaction form, so that the user get only his\hers VMs and only the business groups the he\she is a member of. At first I thought I could do the input validation in the main ASD request form, so that values in the specific field would be linked to an external source (vRO action). However I did not know how to get this to work, because I dont know of a way to extract the __asd_requestUser variable value before the workflow is started, hence the input validation, like I would want to have it, will not work. Thus, I have decided to get the user inputs later in the user interaction form.

[SeanKohler]

Yeah this will only work in VRO.  It is the wrong operational pattern for ASD.

The workflow doesn't EXECUTE until after the form is SUBMITTED from ASD.  At that time the form is gone to the user and a user interaction isn't sent back to the user.  You are probably timing out in VRA because the Get Input Parameter user interaction is never answered in VRO.

You need to pull all the Input parameter work into the INPUT of the called workflow and supply the values in the inputs from vRO Action calls.  (e.g. have a Get VMs Action,  have a get BGs action)

The workflow that you are setting up to be called from ASD is your FORM. The presentation layer of THAT called workflow is the magic that needs to happen to present your users the things you want them to see.  (this capability is extended by capabilities in vRA ASD to call vRO actions directly... but that isn't as important to your operational understanding as the above)

If I have some time today, I will flip some of this around and show you the difference.  Good news is that we can get you there, no problem.  You have some pretty advanced concepts going here.

[SeanKohler]

Sorry.  I guess you are trying to do the presentation all through manual user action in vRA.  That is something I played around with a lot less that getting the data up front in the presentation of the ASD form, but I see where you are going with that now.  The presentation work would more or less be the same on the VRO end.  It still might be useful to pull an action that gets the user owned attributes into an action up front.  I am curious if there a distinct reason you pulled it into manual user action.

[SeanKohler]

I attached three workflows and an action that shows a few different ways of the VirtualMachine Owner lists.  (all in string arrays)

1. The first one is sort of like you are doing things. (only leveraging strings as I discussed earlier) It uses a manual user action.

2. The second one pulls in an action instead of a script/workflow to get the string list. It uses a manual user action.

3. The third just puts the action right into the presentation and therefore looks up the owned VMs when the INITIAL form is run (ASD form presentation, but via vRO OGNL)  There is another way to do this one and that would be to call the action straight from the ASD form and bypass the presentation in the vRO workflow.  (there are reasons why this might not be a good idea)

They are all pretty fast and should run as they exist without modification. (exception being if you have multiple vcachosts... some requests will fail on submit)  You can look at the load time in your sandbox environment using the first workflow.  10 owned machines runs in about 2-3 seconds for me.

All that said: Your slowness is probably due to business groups entirely. I didn't have time today to work on that, but the approach might be to go around the plugin with direct REST calls.  I can put some examples together tomorrow.  The way that you did User ldap lookup is outstanding! I have actually learned a lot from you while looking into this. So this is me giving back.

Please take a look at the attached methods... and I will see if I can help you improve your "Requestor is in these Business Groups" slowness.

Here are four screenshots relative to workflow number 3  (the ASD form side of things).

[SeanKohler]

Hi,

I am looking at this some more today, but I had a thought.

If you had the owned VMs of the requestor, do you really need the business groups outside of the VMs... even if the requestor is in other business groups?

So if I am in BG A, B & C, but I have a machine in BG A & C, couldn't you only present BG A & C in the form?  Because that would be where I would be creating firewall rules, etc.

I ask, because it is VERY easy to get the Business Group name for selected machines.

catalogResource.getOrganization().getSubtenantLabel()

So if your practice was to have the requestor pick the machines they own that need firewall rules, you can just get the Business Group those machines are part of to satisfy your later workflows. (they wouldn't need to select the Business Group because the machine lives in it)

This way you could skip the super parsing required for getting the Business Groups I am part of, because it is only important the Business Groups I am part of where I actually own machines that need to be updated.

That said... I still want a fast way to get the Business Groups a user is part of.  In our case it is really easy because we created one to one mappings of the BG roles to AD groups (AD group has the same name as the Business Group).  If I want to know if a user is in a business group, I just do a recursive lookup in the AD group for the user (local or nested).  It is very fast.  So I could use the ldap lookup on the user like you presented to get his groups and then piece together a name list without ever needing to touch an actual Business Group Object.

I understand that doesn't work for others who are doing their Access Management within the Business Group roles, but there are maybe big changes coming to Business Groups and Authentication... so people would possibly want to be careful.

[Czernobog]

So if your practice was to have the requestor pick the machines they own that need firewall rules, you can just get the Business Group those machines are part of to satisfy your later workflows. (they wouldn't need to select the Business Group because the machine lives in it)

This way you could skip the super parsing required for getting the Business Groups I am part of, because it is only important the Business Groups I am part of where I actually own machines that need to be updated.

That is not possible in this constellation. The only workflow input I get, is the name (string) of the workflow user, passed on by vRA. There is an VCACCAFE:User object provided by the plug-in, but I don't really know how to use it - I did not find a way to link the username I get to this object. Even If there an "user" object I could use, there is no mapping to the vm's, no such thing as "vCACCAFEUser.getOwnedVms()" or something similar. A method like this is really missing and not only that, stuff like vCACCAFEUser.getParentBusinessGroup, businessGroup.getAllCatalogResources or businessGroup.getvCACCAFEHost would help in speeding up the process of getting all relevant information. No easy way of linking together all those relevant objects in vRA is the reason, the workflow is so convoluted and that you have to use slow methods like Server.findAllForType etc., and then iterate through each object, to get all the information you need. An additional chalenge in my case was, that the business group users can be entered as AD Groups, so each time I have to check each role member array (admins, support, users), if there is an AD group present (using primitive placeholder regex for now) and check inside to see if the workflow user is part of it.

Maybe some new useful stuff got included in vRO 7.0, which was released yesterday?

I have also tested what you have proposed previously, that is, loading only arrays of simple objects (strings) into the user presentation, but there was no improvement.

You need to pull all the Input parameter work into the INPUT of the called workflow and supply the values in the inputs from vRO Action calls.  (e.g. have a Get VMs Action,  have a get BGs action)

The reason why I try it with the user interaction is, that the external vRO actions, which would load the business gorups and catalog reosurces, take painfully long to complete, timing out the presentation\form just the same as in the case with the user interaction. I thought that, if I generated my arrays first and loaded the presentation AFTER it is done, I would just only have to fill in the form fields with the done arrays, so the execution would be much faster (no calculation in the form field like with external action).

I did not test oyur workflows yet, the last 3 days we had to fight with a vRA bug, where the SSO solution users in vRA suddenly disappeared, leaving us with no access to vRA... as soon as the issue is resolved, I will get to optimize the workflows.

Thank you so far for all the input!

[SeanKohler]

>>>there is no mapping to the vm's, no such thing as "vCACCAFEUser.getOwnedVms()"

Yes... that is what I wrote for you in the three workflows and one action above.  It maps user@domain.com (the requestor) to all his owned virtual machines (as CatalogResources: which is the Café side of the three-part virtual machine object: VCVM, CafeVM, IaaSVM). So since you would have all machines owned by that person, you can get the business groups that each of the machines are in with one line of code.

>>Maybe some new useful stuff got included in vRO 7.0, which was released yesterday?

Yes. Business Groups have been moved.  I anticipate a speed increase in the new plugins.

>>The reason why I try it with the user interaction is, that the external vRO actions, which would load the business gorups and catalog reosurces, take painfully long to complete,

Yes.  This is brilliant. You taught me something new.  Thank you!

>>> I did not test oyur workflows yet,

Highly recommend it.  Might be a light-bulb.

>>Thank you so far for all the input!

And thank you because I like to talk to other people working through hard challenges.

[SeanKohler · Accepted solution]

Ok, so observe the screenshots and the workflow in this post very carefully.  The speed cannot be denied, and no matter what VMware comes up with relative to getting objects (as the platform is currently written), I think this methodology will be faster in presentation.

What you will see in this workflow:

Getting a requestor's owned VMs is fast and easy through the IaaS objects (where OWNER is maintained): ~5 seconds per 140 total machines / 10 owned machines

user variable is username@domain.com format... e.g. the format of a VRA requestor

var entities = Server.findAllForType("VCAC:VirtualMachine");
//If you are dealing with one vcacHost... otherwise you have to bake this into the loop per ent instance
var vcacHost = Server.findForType("vCAC:VCACHost", entities[0].getEntity().hostId);
for each (ent in entities){
 if (ent.isManaged == true && ent.getEntity().getLink(vcacHost,"Owner")[0].getProperty("UserName") == user){
  virtualMachineNames.push(ent.displayName);
  catalogResources.push(vCACCAFEEntitiesFinder.findCatalogResources(cafeHost,ent.displayName)[0]);
 }
}

Once all the machines are in the array, getting a deduplicated list of Business Groups that support the machines is fast: .02 seconds for 10 machines

for each (catalogResource in catalogResources){
 try{
  tempLabel = catalogResource.getOrganization().getSubtenantLabel();
  } catch (err) {
   tempLabel = null; 
  }
 if ( tempLabel != null && businessGroupNames.indexOf(tempLabel) == -1 ){
  businessGroupNames.push(tempLabel);
  bgcount++;
 }
}

The massive time differences in working with the EntitiesFinder methods for BusinessGroups between FIND and GET: ~24seconds vs ~2seconds to get the same BG.

//This is slow... but that is okay.  It ISN'T slow when the user is selecting things.  
// It is slow in post-user workflow.  If we carry the ID through for the business group
// we can increase the speed of this through the GET section of the entities finder or via REST
var start, finish;
start = Date.now();
selectedBusinessGroup = vCACCAFEEntitiesFinder.findBusinessGroups(cafeHost,mySelectedBusinessGroup)[0];
finish = Date.now();
System.log(selectedBusinessGroup.getName() + " Cafe Business Group was selected and found.");
System.log("Total time to find that one BG: " + (finish-start)/1000 + " seconds ");

// How fast is it with the ID?
var tempId = selectedBusinessGroup.getId(); // let's get the id
start = Date.now();
var getProof = vCACCAFEEntitiesFinder.getBusinessGroup(cafeHost,tempId); // get the object again with the ID
finish = Date.now();
System.log ("Using Get method with ID it took " + (finish-start)/1000 + " seconds to get the BG");

Keep in mind here... the form that you see below came up in 5 seconds after workflow start with the data.

It has a list of Owned VMs and a list of Business Groups.  It isn't exactly what you are looking to do, but provides a couple of faster methods than you are using now.

For the workflow, you will need to edit it and bind your Café Host to the Attribute.

[Czernobog]

Hi,

I have finally gotten to test your approach. Indeed it is much faster, here are my testresults:

I will leave the input fields in the user interaction, because of the customizing that has to be done. I still need to include the business groups that the requestor is a member of, even when there are no owned vm's and also make it able to run on more hosts. This will all extend the form loading time. Still, I bet this approach will be faster that iteating thorugh the Cafe objects.

The biggest takeaways are: working with simple objects in presentations and manipulating vcac entity objects instead of the cafe objects. Also - Principal ID as default form value. In the past I have tried to use the Principal ID as the default form value, however I did not know to what object type I can attach it too. I've looked at the Cafe User object, but it still made no sense.

I still have one concern though. In addition to general slownes of loading the user interaction, another reason why I took the approach you have seen, was the fear, that rather sooner than later using the vcac entity model will not be a viable way to write workflows. From what I know is, that more and more functionality is put into the Cafe component, and at some point the vCAC (IaaS) part will be obsolete (in the context of workflow programming). Another reason would be, that exploring the Cafe objects is simpler, thanks to the API documentation in vRO, and this approach is the one which most people choose, I guess, because it is more accessible.

Thank you again for all the input, it really changed the way I look at the workflow execution.

[SeanKohler]

>>>I still have one concern though. In addition to general slownes of loading the user interaction, another reason why I took the approach you have seen, was the fear, that rather sooner than later using the vcac entity model will not be a viable way to write workflows. From what I know is, that more and more functionality is put into the Cafe component, and at some point the vCAC (IaaS) part will be obsolete (in the context of workflow programming). Another reason would be, that exploring the Cafe objects is simpler, thanks to the API documentation in vRO, and this approach is the one which most people choose, I guess, because it is more accessible.

-------------------

I share your concern, and I struggle with this.  I came to the decision that I will try to make actions and workflows modular enough to be able to switch out certain methods when upgrades force me to do so (like I built a getBusinessGroupName and getProvisioningGroupName actions, for example).  In many ways, it will be difficult to get around a need to change... you can try to jump on a perceived *future* model and the future can turn right in front of you.  (Like Business Groups moving to the CAFE, and I bet the plugin will have new methods.)

VMware engineering has expressed that we should try to use CloudClient for applicable tasks against the API in order to absorb some of these change impacts as time goes on. So for a lot of our execution based stuff, we are calling out to a script host with variable data against consistent command structures (noun/verb).  Time will tell if this is a worthwhile path for us.

Hopefully you will share more of your excellent work as you go along.  I really appreciate seeing the challenges others face, and the creative ways they get around these challenges.