VMware Cloud Community
BenderTheGreate
Contributor
Contributor

Finding a VM via a VcSdkConnection sometimes does not carry over to other elements

We are having an issue where I find a VM using a VcSdkConnection returned from VcPlugin.allSdkConnections, but the VM object that gets returned is null in another element. This code works fine most of the time, but intermittently the returned VcVirtualMachine object does not survive and becomes null in subsequent workflow elements. This continues for a while (about 30 or so minutes) before it starts behaving again. I am at a loss for what is happening, any assistance to track down the issue is greatly appreciated.

17 Replies
qc4vmware
Virtuoso
Virtuoso

If anyone from VMware answers they will likely want to know your vRO version and vSphere plugin version.

Do you have any sample code you provide?  Are you sure you are always using the correct sdkConnection?  If you are finding the vm by cycling through the array of what is returned by VcPlugin.allSdkConnections you may be hitting an issue where the array element does not contain the connection you are expecting.  I don't believe order is guaranteed but I could be wrong.

0 Kudos
BenderTheGreate
Contributor
Contributor

Sorry, very good point!

vRO Version: 7.2.1

VcPlugin Version: 7.2.1.5097044

Here is the code of the action which grabs the VcVirtualMachine object:

// Input Parameters
// vCACVm - VCAC:VirtualMachine

// Return Type - Vc:VirtualMachine

var vCenterVm;

var sdkConnections = VcPlugin.allSdkConnections;

System.debug(sdkConnections.length + " sdk Connections found...");

System.debug("vmUniqueID: " + vCACVm.vmUniqueID);

for each (var sdkConnection in sdkConnections) {

     try {

          vCenterVm = sdkConnection.searchIndex.findByUuid(null, vCACVm.vmUniqueID, true, false);

     } catch(e) {System.error("Error for SDK connection " + sdkConnection.name + " : " + e);}

     if (vCenterVm != null) {

          System.log("Resolved vCenter VM " + vCenterVm.name);

          return vCenterVm;

     }

     try {

          vCenterVm = sdkConnection.searchIndex.findByDnsName(null, vCACVm.vmDNSName, true);

     } catch(e) {System.error("Error for SDK connection " + sdkConnection.name + " : " + e);}

     if (vCenterVm != null) {

          System.log("Resolved vCenter VM " + vCenterVm.name);

          return vCenterVm;

     }

     try {

          var allVms = sdkConnection.getAllVirtualMachines();

          for (var i in allVms) {

               //System.debug(" " + allVms[i].name + ":" + allVms[i].config.instanceUuid);

               if (isSet(allVms[i].config) && isSet(allVms[i].config.instanceUuid)) {

                    if (allVms[i].config.instanceUuid == vCACVm.vmUniqueID) {

                         vCenterVm = allVms[i];

                         System.log("Resolved vCenter VM " + vCenterVm.name);

                         return vCenterVm;

                    }

               }

          }

     } catch(e) {System.log("Error for SDK connection " + sdkConnection.name + " : " + e);}

}

function isSet(s) {

     return s!=null && s!=undefined && s!="";

}

0 Kudos
BenderTheGreate
Contributor
Contributor

As for iterating over VcPlugin.allSdkConnections, we only have one vCenter server configured at this time.

0 Kudos
qc4vmware
Virtuoso
Virtuoso

Sounds to me like you might have either a connectivity issue between the vRO server and the vCenter server or maybe be hitting a bug of some sort.  Is there any reason you can't run 7.3?  There is also a tech preview of the vSphere plugin which I think is compatible with your version of vRO.  You might try upgrading and see if the issue subsides.

Have you scanned through the logs when the error pops up?  Anything interesting show up?

BenderTheGreate
Contributor
Contributor

Scanning through the scripting log, there is nothing out of sorts. We don't have Loginsight in this environment yet, but in our development environment we see TRACE level logging in Loginsight. I was hoping to see some trace logs here that might indicate what is going on.

0 Kudos
BenderTheGreate
Contributor
Contributor

Also, I was mistaken. The code is NOT resolving the VM in the first place (I misinterpreted the log at first), though the VM does exist in the vCenter inventory.

0 Kudos
qc4vmware
Virtuoso
Virtuoso

What version of vCenter is running?

0 Kudos
qc4vmware
Virtuoso
Virtuoso

Also, can you log some information on the vCAC vm?  Are you sure you aren't dealing with a null value there? If this code is being called during cleanup or something like that you might have a race condition where sometimes its called while the objects exist and sometimes one or more are gone.  In our environment we have nearly 20K vm's across 4 vCenters and look the vms up by uuid thousands of times a day without failure.

0 Kudos
BenderTheGreate
Contributor
Contributor

The only field off of the vCAC:VirtualMachine we use in the action is the vmUniqueId property, which we log out and has a valid value. We also log out every property associated with the request and VM to the debug log when the workflow is first called, nothing is null that shouldn't be. I suppose it's possible that the vmUniqueId is somehow not matching the Vc:VirtualMachine's instanceUuid within vCenter, but our process does not log out each VM it compares to in vCenter. Might be worth modifying the action to output what it's comparing to each time, so we can understand if the VM is found but the UUIDs are mismatched.

0 Kudos
BenderTheGreate
Contributor
Contributor

Also, we are running vCenter 6.5.

0 Kudos
qc4vmware
Virtuoso
Virtuoso

When you say you can see the vm in the inventory are you looking at the Orchestrator inventory?  How is the plugin configured if it is per user try flipping it to an account you are 100% sure has access to view the inventory.

0 Kudos
qc4vmware
Virtuoso
Virtuoso

I'd also confirm that all the services are in fact functioning on the vCenter server.  We've had all sorts of issues with the more recent vCenters where a service or a handful is in a bad state and it causes all sorts of weird behavior... sometime gui is broken and orchestrator works and the reverse.

0 Kudos
BenderTheGreate
Contributor
Contributor

The action code is above, but basically it first uses the SDK Connection to find by UUID. If that fails, it then searches by DNS name. If that STILL fails, we pull the VM inventory into Orchestrator with sdkConnection.getAllVirtualMachines() and try to match the Vc:VirtualMachine instanceUuid with the vCAC:VirtualMachine vmUniqueId. In the past, if the first two calls fail (and sometimes they do), the third method using getAllVirtualMachines() was working every time.

The vCenter Server connection is configured with a shared session, not per user, and is configured using a workflow from the VcPlugin. This user has the appropriate permissions to see the VM inventory.

0 Kudos
qc4vmware
Virtuoso
Virtuoso

Did you confirm, from within the vRO client, you can browse to the vm?  If so I'm kind of out of suggestions.  Try rebooting both the vRO server and the vCenter server to see if it magically starts working again.

0 Kudos
BenderTheGreate
Contributor
Contributor

Just realized I never updated back here. We finally got hold of an updated VcPlugin and it fixed the issue.

qc4vmware
Virtuoso
Virtuoso

Can you mark one of my answers as "helpful" so I get some points on they system here?  I'm trying to level up!  Thanks!

0 Kudos
BenderTheGreate
Contributor
Contributor

Done, did the one where you suggested a bug, which since updating fixed the issue, I assume it was.