vRA Software Components failing with Download Error

[pezh0re]

I'm trying to use Software Components to download and install a client util on a Windows Server 2016 guest OS. I first started with a large PowerShell script that would download, install, and configure the application, but have pared it down/chunked it out into smaller steps to see if I can determine where my failure occurs.

Even the most simple batch script to create a folder if it doesn't exist is causing this error during deployment:

The following component requests failed: vSphere__vCenter__Machine_1. Request failed: Machine TestVM1: InstallSoftware : Exception encountered: Exception calling "DownloadFile" with "2" argument(s): "The underlying connection was closed: An unexpected error occurred on a receive."[0].Exception and TestVM1: sendEBSMessage4(workflow=04d3e31c-7bbc-4113-8bfc-60822786edb6) Error in state VMPSMasterWorkflow32.Requested phase PRE event (queue = 4d3aeb89-4ec6-46eb-b47e-fd8f09a9cf9d): Extensibility consumer error(20999) - TypeError: Cannot read property "VMPSMasterWorkflow32" from undefined (Dynamic Script Module name : ObjectWrapper#7).

vRA logs show the following entries:

TestVM1: InstallSoftwareWorkflow Exception: WorkItem response indicates proxy agent failed to perform task.

DynamicOps.Common.WorkItems.WorkItemResponseException: WorkItem response indicates proxy agent failed to perform task.

at DynamicOps.External.SoftwareWorkflows.InstallSoftwareWorkflow.Error_SendWorkitem(Object sender, EventArgs e)

at System.Workflow.ComponentModel.Activity.RaiseEvent(DependencyProperty dependencyEvent, Object sender, EventArgs e)

at System.Workflow.Activities.CodeActivity.Execute(ActivityExecutionContext executionContext)

at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext)

at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)

at System.Workflow.Runtime.Scheduler.Run()

TestVM1: InstallSoftwareWorkflow SendWorkitem Exception: WorkItem response indicates proxy agent failed to perform task.

DynamicOps.Common.WorkItems.WorkItemResponseException: WorkItem response indicates proxy agent failed to perform task.

at DynamicOps.VMPS.Workflow.Activities.ReceiveWorkItemResponseActivity.OnInvoked(EventArgs e)

at System.Workflow.Activities.HandleExternalEventActivity.RaiseEvent(Object[] args)

at System.Workflow.Activities.HandleExternalEventActivity.Execute(ActivityExecutionContext executionContext)

at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext)

at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)

at System.Workflow.Runtime.Scheduler.Run()

Has anyone run into this issue in the past?

I would include additional guest agent logs, but unfortunately vRA is destroying the VMs when the build fails - even with the _debug_deployment blueprint property set to true.

I'm on vRA version 7.3.0 (build 5604410).

[daphnissov]

This message indicates there is an infrastructure or connection-related problem between the template/deployed machine and vRA. Some common reasons are the guest/software agent wasn't installed correctly, certificates aren't right, firewall, and some others. Also the _debug_deployment custom property only comes in to play during the actual software component's execution. It isn't even getting to that point, so that's why vRA is destroying the machine regardless of that property's presence or not.

Can you first start out by describing your vRA architecture and the license type you have assigned? Next, go to your template and use the PS1 script to uninstall and then re-install the agent but screenshotting the process so we can see the values you're providing. These are a few steps, but it's tricky to troubleshoot.

[pezh0re]

You're probably correct that this is an agent issue - we have Linux servers deployed to the same network and those builds can perform software component installation. Since we're not using NSX/OS-specific filters I can only assume Windows VMs on the same IP space/network would have the same network connectivity.

It's good to know that the _debug_deployment only affects builds that have proceeded into the software component portion - I'll keep that in mind.

Evidently we have vRA 7.3 Enterprise with an Enterprise deployment (load balancers, two Appliances, IaaS web, and IaaS Manager services, etc).

I'm not directly responsible for the underlying virtual infrastructure here, but I'll ask those who are to uninstall/reinstall the agent and take screenshots.